[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:16:55.061594 15476 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.29.62:40789
I20260709 13:16:55.063238 15476 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:16:55.064339 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:55.081473 15483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.081462 15482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:55.081501 15485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:55.082365 15476 server_base.cc:1061] running on GCE node
I20260709 13:16:56.284796 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:56.285106 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:56.285329 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603016285307 us; error 0 us; skew 500 ppm
I20260709 13:16:56.286207 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:56.292104 15476 webserver.cc:533] Webserver started at http://127.15.29.62:37517/ using document root <none> and password file <none>
I20260709 13:16:56.293093 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:56.293367 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:56.293853 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:56.299923 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/master-0-root/instance:
uuid: "9e758ef2a0cd4885965a7bf07143b8ab"
format_stamp: "Formatted at 2026-07-09 13:16:56 on dist-test-slave-hz8m"
I20260709 13:16:56.308081 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:16:56.313458 15491 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:56.314666 15476 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:16:56.315112 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/master-0-root
uuid: "9e758ef2a0cd4885965a7bf07143b8ab"
format_stamp: "Formatted at 2026-07-09 13:16:56 on dist-test-slave-hz8m"
I20260709 13:16:56.315553 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:56.370342 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:56.372026 15476 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:16:56.372530 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:56.447144 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.62:40789
I20260709 13:16:56.447222 15542 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.62:40789 every 8 connection(s)
I20260709 13:16:56.455399 15543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:56.477507 15543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab: Bootstrap starting.
I20260709 13:16:56.483566 15543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:56.485443 15543 log.cc:826] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab: Log is configured to *not* fsync() on all Append() calls
I20260709 13:16:56.490124 15543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab: No bootstrap required, opened a new log
I20260709 13:16:56.510984 15543 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" member_type: VOTER }
I20260709 13:16:56.511514 15543 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:56.511715 15543 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e758ef2a0cd4885965a7bf07143b8ab, State: Initialized, Role: FOLLOWER
I20260709 13:16:56.512516 15543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" member_type: VOTER }
I20260709 13:16:56.512974 15543 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:16:56.513216 15543 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:16:56.513499 15543 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:56.517482 15543 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" member_type: VOTER }
I20260709 13:16:56.518188 15543 leader_election.cc:304] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e758ef2a0cd4885965a7bf07143b8ab; no voters: 
I20260709 13:16:56.519975 15543 leader_election.cc:290] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:16:56.520326 15546 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:56.522662 15546 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 1 LEADER]: Becoming Leader. State: Replica: 9e758ef2a0cd4885965a7bf07143b8ab, State: Running, Role: LEADER
I20260709 13:16:56.523722 15546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" member_type: VOTER }
I20260709 13:16:56.524850 15543 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:16:56.534309 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" member_type: VOTER } }
I20260709 13:16:56.534566 15548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e758ef2a0cd4885965a7bf07143b8ab. Latest consensus state: current_term: 1 leader_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e758ef2a0cd4885965a7bf07143b8ab" member_type: VOTER } }
I20260709 13:16:56.535133 15548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:56.535133 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [sys.catalog]: This master's current role is: LEADER
I20260709 13:16:56.538295 15555 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:16:56.550361 15555 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:16:56.552599 15476 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:16:56.568183 15555 catalog_manager.cc:1383] Generated new cluster ID: 8aeab4dc6140481fb88275627814e223
I20260709 13:16:56.568509 15555 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:16:56.592414 15555 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:16:56.593990 15555 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:16:56.608385 15555 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab: Generated new TSK 0
I20260709 13:16:56.609416 15555 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:16:56.619807 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:56.630851 15565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:56.632477 15566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:56.638823 15476 server_base.cc:1061] running on GCE node
W20260709 13:16:56.639292 15568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:56.640169 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:56.640404 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:56.640625 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603016640607 us; error 0 us; skew 500 ppm
I20260709 13:16:56.641214 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:56.645780 15476 webserver.cc:533] Webserver started at http://127.15.29.1:32819/ using document root <none> and password file <none>
I20260709 13:16:56.646384 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:56.646620 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:56.646978 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:56.648188 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-0-root/instance:
uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
format_stamp: "Formatted at 2026-07-09 13:16:56 on dist-test-slave-hz8m"
I20260709 13:16:56.656364 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.001s	sys 0.007s
I20260709 13:16:56.662629 15573 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:56.663535 15476 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:16:56.663847 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-0-root
uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
format_stamp: "Formatted at 2026-07-09 13:16:56 on dist-test-slave-hz8m"
I20260709 13:16:56.664136 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:56.683979 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:56.685393 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:56.688397 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:56.693233 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:56.693475 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:56.693796 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:56.693969 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:56.804952 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.1:45219
I20260709 13:16:56.805049 15635 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.1:45219 every 8 connection(s)
I20260709 13:16:56.813128 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:56.823431 15640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:56.834398 15641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:56.841703 15636 heartbeater.cc:344] Connected to a master server at 127.15.29.62:40789
I20260709 13:16:56.842262 15636 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:56.843564 15636 heartbeater.cc:507] Master 127.15.29.62:40789 requested a full tablet report, sending...
W20260709 13:16:56.844240 15643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:56.845351 15476 server_base.cc:1061] running on GCE node
I20260709 13:16:56.846395 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:56.846657 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:56.846918 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603016846896 us; error 0 us; skew 500 ppm
I20260709 13:16:56.847111 15508 ts_manager.cc:194] Registered new tserver with Master: 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219)
I20260709 13:16:56.847713 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:56.850083 15508 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37826
I20260709 13:16:56.855444 15476 webserver.cc:533] Webserver started at http://127.15.29.2:36635/ using document root <none> and password file <none>
I20260709 13:16:56.856045 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:56.856267 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:56.856552 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:56.857671 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-1-root/instance:
uuid: "0ad934ae28874ff083d1c1305fc718d1"
format_stamp: "Formatted at 2026-07-09 13:16:56 on dist-test-slave-hz8m"
I20260709 13:16:56.863672 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.005s
I20260709 13:16:56.868016 15648 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:56.868956 15476 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20260709 13:16:56.869274 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-1-root
uuid: "0ad934ae28874ff083d1c1305fc718d1"
format_stamp: "Formatted at 2026-07-09 13:16:56 on dist-test-slave-hz8m"
I20260709 13:16:56.869604 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:56.900065 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:56.901412 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:56.903683 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:56.907187 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:56.907425 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:56.907694 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:56.907869 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:57.011548 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.2:35253
I20260709 13:16:57.011682 15710 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.2:35253 every 8 connection(s)
I20260709 13:16:57.025621 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:16:57.034493 15714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:16:57.037886 15715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:57.041551 15711 heartbeater.cc:344] Connected to a master server at 127.15.29.62:40789
I20260709 13:16:57.041967 15711 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:57.042742 15711 heartbeater.cc:507] Master 127.15.29.62:40789 requested a full tablet report, sending...
W20260709 13:16:57.044708 15717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:16:57.045554 15508 ts_manager.cc:194] Registered new tserver with Master: 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:57.045660 15476 server_base.cc:1061] running on GCE node
I20260709 13:16:57.047211 15508 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37840
I20260709 13:16:57.047824 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:16:57.048107 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:16:57.048310 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603017048289 us; error 0 us; skew 500 ppm
I20260709 13:16:57.048934 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:16:57.051867 15476 webserver.cc:533] Webserver started at http://127.15.29.3:44873/ using document root <none> and password file <none>
I20260709 13:16:57.052376 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:16:57.052577 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:16:57.052973 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:16:57.054118 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-2-root/instance:
uuid: "ebaa234beb6544fc9c9151e623044058"
format_stamp: "Formatted at 2026-07-09 13:16:57 on dist-test-slave-hz8m"
I20260709 13:16:57.059033 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:16:57.062517 15722 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:57.063493 15476 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:16:57.063813 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-2-root
uuid: "ebaa234beb6544fc9c9151e623044058"
format_stamp: "Formatted at 2026-07-09 13:16:57 on dist-test-slave-hz8m"
I20260709 13:16:57.064126 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783603014893117-15476-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:16:57.097648 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:16:57.098974 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:16:57.101083 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:16:57.104398 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:16:57.104569 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:57.104880 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:16:57.105046 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:16:57.203490 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.3:46095
I20260709 13:16:57.203585 15784 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.3:46095 every 8 connection(s)
I20260709 13:16:57.220820 15785 heartbeater.cc:344] Connected to a master server at 127.15.29.62:40789
I20260709 13:16:57.221185 15785 heartbeater.cc:461] Registering TS with master...
I20260709 13:16:57.221938 15785 heartbeater.cc:507] Master 127.15.29.62:40789 requested a full tablet report, sending...
I20260709 13:16:57.223892 15508 ts_manager.cc:194] Registered new tserver with Master: ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:57.224014 15476 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015326826s
I20260709 13:16:57.225545 15508 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37844
I20260709 13:16:57.277613 15508 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37858:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:16:57.281255 15508 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:16:57.402244 15749 tablet_service.cc:1511] Processing CreateTablet for tablet 8dda50be1bbe4c698a0bab707d89017a (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:57.407608 15749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dda50be1bbe4c698a0bab707d89017a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.402271 15750 tablet_service.cc:1511] Processing CreateTablet for tablet 701ac14f84214ca6a9ff6c7be2291ef2 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:57.407536 15748 tablet_service.cc:1511] Processing CreateTablet for tablet 5b6fb9cb427a4e92a9931b6f98a88548 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:57.407544 15747 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3ddd6a760b4ce18907522e6e096ff3 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:57.411383 15750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701ac14f84214ca6a9ff6c7be2291ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.413673 15748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b6fb9cb427a4e92a9931b6f98a88548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.418103 15747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3ddd6a760b4ce18907522e6e096ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.420295 15676 tablet_service.cc:1511] Processing CreateTablet for tablet 701ac14f84214ca6a9ff6c7be2291ef2 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:57.421834 15676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701ac14f84214ca6a9ff6c7be2291ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.421775 15675 tablet_service.cc:1511] Processing CreateTablet for tablet 8dda50be1bbe4c698a0bab707d89017a (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:57.423331 15675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dda50be1bbe4c698a0bab707d89017a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.424257 15673 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3ddd6a760b4ce18907522e6e096ff3 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:57.425766 15673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3ddd6a760b4ce18907522e6e096ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.428268 15672 tablet_service.cc:1511] Processing CreateTablet for tablet 9612e9531a6343ef9af8c5ca01d50c61 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:57.429771 15672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9612e9531a6343ef9af8c5ca01d50c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.431228 15674 tablet_service.cc:1511] Processing CreateTablet for tablet 5b6fb9cb427a4e92a9931b6f98a88548 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:57.436187 15674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b6fb9cb427a4e92a9931b6f98a88548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.440514 15746 tablet_service.cc:1511] Processing CreateTablet for tablet 9612e9531a6343ef9af8c5ca01d50c61 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:57.442286 15746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9612e9531a6343ef9af8c5ca01d50c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.466266 15600 tablet_service.cc:1511] Processing CreateTablet for tablet 8dda50be1bbe4c698a0bab707d89017a (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:16:57.477530 15600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dda50be1bbe4c698a0bab707d89017a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.465497 15601 tablet_service.cc:1511] Processing CreateTablet for tablet 701ac14f84214ca6a9ff6c7be2291ef2 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:16:57.482438 15596 tablet_service.cc:1511] Processing CreateTablet for tablet ea0971e8177a4bf99eb967030baf8483 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:57.482471 15599 tablet_service.cc:1511] Processing CreateTablet for tablet 5b6fb9cb427a4e92a9931b6f98a88548 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:16:57.484136 15671 tablet_service.cc:1511] Processing CreateTablet for tablet ea0971e8177a4bf99eb967030baf8483 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:57.483431 15598 tablet_service.cc:1511] Processing CreateTablet for tablet 0c3ddd6a760b4ce18907522e6e096ff3 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:16:57.490309 15671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea0971e8177a4bf99eb967030baf8483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.488004 15597 tablet_service.cc:1511] Processing CreateTablet for tablet 9612e9531a6343ef9af8c5ca01d50c61 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:16:57.492424 15601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701ac14f84214ca6a9ff6c7be2291ef2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.504648 15598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c3ddd6a760b4ce18907522e6e096ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.506584 15597 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9612e9531a6343ef9af8c5ca01d50c61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.511113 15599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b6fb9cb427a4e92a9931b6f98a88548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.509438 15803 tablet_bootstrap.cc:492] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:57.515045 15596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea0971e8177a4bf99eb967030baf8483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.532771 15750 tablet_service.cc:1511] Processing CreateTablet for tablet ea0971e8177a4bf99eb967030baf8483 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:16:57.533794 15750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea0971e8177a4bf99eb967030baf8483. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:16:57.547700 15556 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:57.567463 15595 tablet_service.cc:1511] Processing CreateTablet for tablet b6ca0bf372de4e199c8636d495a01040 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:57.569337 15595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ca0bf372de4e199c8636d495a01040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.583946 15747 tablet_service.cc:1511] Processing CreateTablet for tablet b6ca0bf372de4e199c8636d495a01040 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:57.586503 15804 tablet_bootstrap.cc:492] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:57.582301 15673 tablet_service.cc:1511] Processing CreateTablet for tablet b6ca0bf372de4e199c8636d495a01040 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:16:57.590019 15594 tablet_service.cc:1511] Processing CreateTablet for tablet 089885ab0d8541f5a1853955cc74c166 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:57.594023 15594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089885ab0d8541f5a1853955cc74c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.594424 15747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ca0bf372de4e199c8636d495a01040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.599647 15746 tablet_service.cc:1511] Processing CreateTablet for tablet 089885ab0d8541f5a1853955cc74c166 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:57.599745 15671 tablet_service.cc:1511] Processing CreateTablet for tablet 089885ab0d8541f5a1853955cc74c166 (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:16:57.601492 15746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089885ab0d8541f5a1853955cc74c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.605305 15675 tablet_service.cc:1511] Processing CreateTablet for tablet 0d1c6bccba264b9b9301ef8b1bde7e8e (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:57.609395 15675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d1c6bccba264b9b9301ef8b1bde7e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.611032 15805 tablet_bootstrap.cc:492] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:57.612370 15671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089885ab0d8541f5a1853955cc74c166. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.613837 15673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6ca0bf372de4e199c8636d495a01040. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.616734 15598 tablet_service.cc:1511] Processing CreateTablet for tablet 0d1c6bccba264b9b9301ef8b1bde7e8e (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:57.617991 15598 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d1c6bccba264b9b9301ef8b1bde7e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.635761 15803 tablet_bootstrap.cc:654] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.642364 15805 tablet_bootstrap.cc:654] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.645087 15750 tablet_service.cc:1511] Processing CreateTablet for tablet 0d1c6bccba264b9b9301ef8b1bde7e8e (DEFAULT_TABLE table=test-workload [id=79e8684635ea4b36bd855d3e97ac7ab7]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:16:57.649159 15750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d1c6bccba264b9b9301ef8b1bde7e8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:16:57.664265 15804 tablet_bootstrap.cc:654] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.668555 15803 tablet_bootstrap.cc:492] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.669092 15803 ts_tablet_manager.cc:1403] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.167s	user 0.015s	sys 0.003s
I20260709 13:16:57.692893 15803 raft_consensus.cc:359] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.693684 15803 raft_consensus.cc:385] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.694027 15803 raft_consensus.cc:740] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.694834 15803 consensus_queue.cc:260] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.695787 15805 tablet_bootstrap.cc:492] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:57.696321 15805 ts_tablet_manager.cc:1403] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.015s
I20260709 13:16:57.698843 15804 tablet_bootstrap.cc:492] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:57.699374 15804 ts_tablet_manager.cc:1403] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.114s	user 0.014s	sys 0.012s
I20260709 13:16:57.699631 15805 raft_consensus.cc:359] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.700439 15805 raft_consensus.cc:385] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.700757 15805 raft_consensus.cc:740] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.701524 15805 consensus_queue.cc:260] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.702381 15785 heartbeater.cc:499] Master 127.15.29.62:40789 was elected leader, sending a full tablet report...
I20260709 13:16:57.704530 15804 raft_consensus.cc:359] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.705631 15804 raft_consensus.cc:385] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.706023 15804 raft_consensus.cc:740] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.709749 15804 consensus_queue.cc:260] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.710816 15636 heartbeater.cc:499] Master 127.15.29.62:40789 was elected leader, sending a full tablet report...
I20260709 13:16:57.708287 15803 ts_tablet_manager.cc:1434] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.039s	user 0.002s	sys 0.007s
I20260709 13:16:57.711669 15805 ts_tablet_manager.cc:1434] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.002s
I20260709 13:16:57.712711 15805 tablet_bootstrap.cc:492] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:57.712711 15803 tablet_bootstrap.cc:492] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
W20260709 13:16:57.715420 15786 tablet.cc:2406] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:16:57.719674 15803 tablet_bootstrap.cc:654] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.719674 15805 tablet_bootstrap.cc:654] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.720971 15711 heartbeater.cc:499] Master 127.15.29.62:40789 was elected leader, sending a full tablet report...
I20260709 13:16:57.724402 15804 ts_tablet_manager.cc:1434] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.005s
I20260709 13:16:57.725723 15804 tablet_bootstrap.cc:492] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:57.731283 15805 tablet_bootstrap.cc:492] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:57.731807 15805 ts_tablet_manager.cc:1403] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:16:57.733114 15804 tablet_bootstrap.cc:654] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.734754 15805 raft_consensus.cc:359] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.735602 15805 raft_consensus.cc:385] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.735930 15805 raft_consensus.cc:740] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.739053 15803 tablet_bootstrap.cc:492] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.739619 15803 ts_tablet_manager.cc:1403] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.027s	user 0.003s	sys 0.009s
I20260709 13:16:57.739820 15805 consensus_queue.cc:260] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.742213 15805 ts_tablet_manager.cc:1434] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:16:57.746892 15805 tablet_bootstrap.cc:492] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:57.749176 15804 tablet_bootstrap.cc:492] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:57.749667 15804 ts_tablet_manager.cc:1403] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.024s	user 0.002s	sys 0.010s
I20260709 13:16:57.744481 15803 raft_consensus.cc:359] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.751099 15803 raft_consensus.cc:385] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.751708 15803 raft_consensus.cc:740] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.752893 15803 consensus_queue.cc:260] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.755520 15803 ts_tablet_manager.cc:1434] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:16:57.756579 15803 tablet_bootstrap.cc:492] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:57.761175 15804 raft_consensus.cc:359] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.761937 15804 raft_consensus.cc:385] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.762311 15804 raft_consensus.cc:740] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.763149 15804 consensus_queue.cc:260] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.762032 15805 tablet_bootstrap.cc:654] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.763973 15803 tablet_bootstrap.cc:654] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.768877 15804 ts_tablet_manager.cc:1434] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:16:57.769836 15804 tablet_bootstrap.cc:492] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:57.778628 15804 tablet_bootstrap.cc:654] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.792654 15805 tablet_bootstrap.cc:492] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:57.793319 15805 ts_tablet_manager.cc:1403] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.047s	user 0.011s	sys 0.011s
I20260709 13:16:57.793515 15803 tablet_bootstrap.cc:492] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.794009 15803 ts_tablet_manager.cc:1403] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.038s	user 0.011s	sys 0.003s
I20260709 13:16:57.794090 15804 tablet_bootstrap.cc:492] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:57.794548 15804 ts_tablet_manager.cc:1403] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 13:16:57.796885 15805 raft_consensus.cc:359] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.797103 15803 raft_consensus.cc:359] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.797787 15805 raft_consensus.cc:385] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.797968 15803 raft_consensus.cc:385] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.797271 15804 raft_consensus.cc:359] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.798280 15805 raft_consensus.cc:740] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.798348 15803 raft_consensus.cc:740] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.798473 15804 raft_consensus.cc:385] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.799172 15804 raft_consensus.cc:740] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.799371 15803 consensus_queue.cc:260] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.799418 15805 consensus_queue.cc:260] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.800058 15804 consensus_queue.cc:260] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.809415 15803 ts_tablet_manager.cc:1434] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.015s	user 0.012s	sys 0.004s
I20260709 13:16:57.810420 15803 tablet_bootstrap.cc:492] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:57.814983 15804 ts_tablet_manager.cc:1434] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.006s
I20260709 13:16:57.817101 15803 tablet_bootstrap.cc:654] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.818774 15804 tablet_bootstrap.cc:492] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:57.823009 15805 ts_tablet_manager.cc:1434] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:16:57.827600 15804 tablet_bootstrap.cc:654] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.828958 15803 tablet_bootstrap.cc:492] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.829504 15803 ts_tablet_manager.cc:1403] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.012s
I20260709 13:16:57.830253 15805 tablet_bootstrap.cc:492] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:57.831244 15811 raft_consensus.cc:493] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.831766 15811 raft_consensus.cc:515] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.833204 15803 raft_consensus.cc:359] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.834187 15803 raft_consensus.cc:385] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.834575 15803 raft_consensus.cc:740] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.835600 15803 consensus_queue.cc:260] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.839226 15803 ts_tablet_manager.cc:1434] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:16:57.840790 15811 leader_election.cc:290] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:57.849041 15803 tablet_bootstrap.cc:492] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:57.852052 15810 raft_consensus.cc:493] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.853315 15810 raft_consensus.cc:515] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.857779 15810 leader_election.cc:290] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:57.860445 15803 tablet_bootstrap.cc:654] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.861371 15805 tablet_bootstrap.cc:654] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.867991 15804 tablet_bootstrap.cc:492] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:57.868686 15804 ts_tablet_manager.cc:1403] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.050s	user 0.031s	sys 0.002s
I20260709 13:16:57.872661 15804 raft_consensus.cc:359] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.876763 15804 raft_consensus.cc:385] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.877283 15804 raft_consensus.cc:740] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.878170 15804 consensus_queue.cc:260] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.880743 15803 tablet_bootstrap.cc:492] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.881021 15805 tablet_bootstrap.cc:492] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:57.881248 15803 ts_tablet_manager.cc:1403] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.010s
I20260709 13:16:57.881639 15805 ts_tablet_manager.cc:1403] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.006s
I20260709 13:16:57.885548 15803 raft_consensus.cc:359] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.886483 15803 raft_consensus.cc:385] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.886796 15803 raft_consensus.cc:740] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.887598 15803 consensus_queue.cc:260] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.888599 15804 ts_tablet_manager.cc:1434] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.005s
I20260709 13:16:57.890105 15803 ts_tablet_manager.cc:1434] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:16:57.891081 15803 tablet_bootstrap.cc:492] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:57.894338 15805 raft_consensus.cc:359] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.895229 15805 raft_consensus.cc:385] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.895524 15805 raft_consensus.cc:740] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.896526 15804 tablet_bootstrap.cc:492] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:57.896260 15805 consensus_queue.cc:260] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.898136 15803 tablet_bootstrap.cc:654] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.899235 15805 ts_tablet_manager.cc:1434] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 13:16:57.900182 15805 tablet_bootstrap.cc:492] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:57.905606 15804 tablet_bootstrap.cc:654] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.907074 15805 tablet_bootstrap.cc:654] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.913604 15803 tablet_bootstrap.cc:492] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.914243 15803 ts_tablet_manager.cc:1403] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 13:16:57.916518 15804 tablet_bootstrap.cc:492] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:57.917083 15804 ts_tablet_manager.cc:1403] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.011s
I20260709 13:16:57.917276 15803 raft_consensus.cc:359] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.918008 15803 raft_consensus.cc:385] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.918326 15803 raft_consensus.cc:740] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.919104 15803 consensus_queue.cc:260] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.920372 15804 raft_consensus.cc:359] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.921350 15804 raft_consensus.cc:385] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.921780 15804 raft_consensus.cc:740] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.922016 15803 ts_tablet_manager.cc:1434] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:16:57.923825 15803 tablet_bootstrap.cc:492] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:57.923321 15804 consensus_queue.cc:260] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.925055 15611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:57.925962 15804 ts_tablet_manager.cc:1434] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:16:57.925875 15611 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 0.
I20260709 13:16:57.926623 15811 raft_consensus.cc:493] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.927103 15811 raft_consensus.cc:515] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.928792 15649 leader_election.cc:304] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:57.930032 15811 leader_election.cc:290] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095), 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219)
I20260709 13:16:57.930567 15811 raft_consensus.cc:2804] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:57.927023 15805 tablet_bootstrap.cc:492] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:57.930965 15811 raft_consensus.cc:493] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.931385 15811 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:57.931907 15804 tablet_bootstrap.cc:492] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:57.933956 15803 tablet_bootstrap.cc:654] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.931361 15805 ts_tablet_manager.cc:1403] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.031s	user 0.004s	sys 0.009s
I20260709 13:16:57.939579 15804 tablet_bootstrap.cc:654] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:57.942490 15805 raft_consensus.cc:359] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.943475 15805 raft_consensus.cc:385] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.943866 15805 raft_consensus.cc:740] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.944761 15805 consensus_queue.cc:260] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:57.946516 15803 tablet_bootstrap.cc:492] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:57.947244 15803 ts_tablet_manager.cc:1403] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.006s
I20260709 13:16:57.951516 15803 raft_consensus.cc:359] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.953105 15803 raft_consensus.cc:385] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:57.953477 15803 raft_consensus.cc:740] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:57.954084 15811 raft_consensus.cc:515] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.954346 15803 consensus_queue.cc:260] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.966262 15809 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.966800 15809 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:57.976991 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:57.977366 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:57.973603 15811 leader_election.cc:290] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:57.982565 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:57.979753 15759 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 0.
I20260709 13:16:57.985714 15805 ts_tablet_manager.cc:1434] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.002s
I20260709 13:16:57.980216 15760 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 0.
I20260709 13:16:57.987562 15651 leader_election.cc:304] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:57.988652 15811 raft_consensus.cc:2804] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:57.989009 15811 raft_consensus.cc:493] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.989267 15805 tablet_bootstrap.cc:492] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:57.989404 15811 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:57.995865 15809 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:57.996730 15809 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.997251 15809 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.003150 15810 raft_consensus.cc:493] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.004552 15810 raft_consensus.cc:515] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:58.004702 15803 ts_tablet_manager.cc:1434] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.013s
W20260709 13:16:58.011319 15649 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:45219, user_credentials={real_user=slave}} blocked reactor thread for 82787.1us
I20260709 13:16:58.023437 15804 tablet_bootstrap.cc:492] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:58.024102 15804 ts_tablet_manager.cc:1403] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.092s	user 0.019s	sys 0.025s
I20260709 13:16:58.040284 15833 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.049353 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.050056 15610 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.030090 15811 raft_consensus.cc:515] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:58.057968 15611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:58.059816 15803 tablet_bootstrap.cc:492] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:58.060112 15611 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 0.
I20260709 13:16:58.072981 15805 tablet_bootstrap.cc:654] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.058152 15833 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.072860 15809 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.062793 15804 raft_consensus.cc:359] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.079470 15811 leader_election.cc:290] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Requested vote from peers ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095), 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219)
I20260709 13:16:58.108075 15833 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.113713 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.114383 15809 raft_consensus.cc:493] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.110598 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.089109 15810 leader_election.cc:290] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.136598 15610 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 1.
I20260709 13:16:58.083314 15804 raft_consensus.cc:385] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.135677 15686 raft_consensus.cc:2393] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 668cfc1d74d2440cacb6b0a6afb2c50e in current term 1: Already voted for candidate 0ad934ae28874ff083d1c1305fc718d1 in this term.
I20260709 13:16:58.140383 15838 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.137560 15611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.136909 15810 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.138809 15685 raft_consensus.cc:2393] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 668cfc1d74d2440cacb6b0a6afb2c50e in current term 1: Already voted for candidate 0ad934ae28874ff083d1c1305fc718d1 in this term.
I20260709 13:16:58.138284 15804 raft_consensus.cc:740] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.144163 15607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:58.142970 15803 tablet_bootstrap.cc:654] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.142886 15611 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.141436 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.140897 15838 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.135692 15809 raft_consensus.cc:515] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.147168 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.143465 15810 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.147548 15759 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.145682 15649 leader_election.cc:304] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: ebaa234beb6544fc9c9151e623044058
I20260709 13:16:58.152024 15811 raft_consensus.cc:2804] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.148552 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:58.148540 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
W20260709 13:16:58.151861 15723 leader_election.cc:343] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.150795 15758 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.158370 15608 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.158334 15609 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.159970 15830 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.149586 15804 consensus_queue.cc:260] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.163336 15838 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.160641 15830 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.165349 15838 raft_consensus.cc:493] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.168016 15810 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.168643 15809 leader_election.cc:290] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.167085 15723 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:58.170555 15838 raft_consensus.cc:515] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.177421 15805 tablet_bootstrap.cc:492] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:58.176347 15684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.179265 15684 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.180939 15805 ts_tablet_manager.cc:1403] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.192s	user 0.016s	sys 0.008s
I20260709 13:16:58.175513 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.183298 15809 raft_consensus.cc:2804] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.186722 15686 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.185207 15575 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.185639 15809 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.192720 15830 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.194165 15830 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.194763 15830 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.197651 15805 raft_consensus.cc:359] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.192890 15809 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.199191 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.200915 15575 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.198607 15805 raft_consensus.cc:385] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.179952 15576 leader_election.cc:304] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:58.203375 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:58.205220 15609 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 0.
I20260709 13:16:58.207654 15839 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.203612 15805 raft_consensus.cc:740] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.192348 15810 raft_consensus.cc:2804] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.199895 15723 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:58.212813 15838 leader_election.cc:290] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.212713 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.215024 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:58.205657 15758 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 0.
I20260709 13:16:58.216442 15649 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.210146 15611 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 1.
I20260709 13:16:58.222419 15649 leader_election.cc:304] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.224512 15838 raft_consensus.cc:2764] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:16:58.225065 15833 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.225142 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.225559 15833 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.225939 15833 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.213770 15830 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.227411 15611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
W20260709 13:16:58.228928 15575 leader_election.cc:343] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.226497 15805 consensus_queue.cc:260] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.231114 15611 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.219274 15810 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.232842 15805 ts_tablet_manager.cc:1434] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
I20260709 13:16:58.256800 15804 ts_tablet_manager.cc:1434] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.232s	user 0.028s	sys 0.022s
I20260709 13:16:58.230540 15830 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.277210 15683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.311450 15830 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.311721 15683 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.309460 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.267426 15684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.312390 15682 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.313212 15724 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:58.313552 15805 tablet_bootstrap.cc:492] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:58.313716 15842 raft_consensus.cc:2804] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.314222 15842 raft_consensus.cc:697] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:58.314543 15724 leader_election.cc:304] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:58.315725 15834 raft_consensus.cc:2804] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.316179 15834 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.316411 15843 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.316596 15834 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.239823 15833 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.318145 15803 tablet_bootstrap.cc:492] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:58.311865 15830 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.318707 15803 ts_tablet_manager.cc:1403] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.270s	user 0.035s	sys 0.047s
I20260709 13:16:58.319123 15810 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.259784 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.322400 15684 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.322198 15803 raft_consensus.cc:359] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.322950 15803 raft_consensus.cc:385] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.325416 15803 raft_consensus.cc:740] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.315114 15842 consensus_queue.cc:237] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:58.325809 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.326440 15608 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.254926 15811 raft_consensus.cc:697] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:58.327135 15803 consensus_queue.cc:260] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.316872 15843 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
W20260709 13:16:58.270124 15576 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:46095, user_credentials={real_user=slave}} blocked reactor thread for 105412us
I20260709 13:16:58.333202 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.241535 15809 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.333636 15843 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.333994 15758 raft_consensus.cc:2393] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 668cfc1d74d2440cacb6b0a6afb2c50e in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:58.335016 15759 raft_consensus.cc:2393] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:58.334904 15684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.336591 15607 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:58.336624 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.337641 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.337893 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.338688 15682 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.340446 15756 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.341184 15756 raft_consensus.cc:2468] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.341329 15757 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.342481 15576 leader_election.cc:304] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:58.343423 15845 raft_consensus.cc:2764] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:16:58.345093 15576 leader_election.cc:304] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:58.345947 15845 raft_consensus.cc:2804] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.356432 15810 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.358273 15805 tablet_bootstrap.cc:654] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.359033 15609 raft_consensus.cc:2393] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 in current term 1: Already voted for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in this term.
I20260709 13:16:58.324957 15833 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.346489 15845 raft_consensus.cc:493] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.348413 15809 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.366335 15811 consensus_queue.cc:237] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.368978 15804 tablet_bootstrap.cc:492] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:58.371074 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
W20260709 13:16:58.374164 15649 leader_election.cc:343] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.373560 15682 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 1.
I20260709 13:16:58.380560 15608 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 1.
I20260709 13:16:58.381122 15839 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.381513 15839 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.347049 15834 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.364293 15845 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.400738 15810 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.429945 15758 raft_consensus.cc:2393] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:58.440816 15846 raft_consensus.cc:2804] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.441433 15846 raft_consensus.cc:493] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.441866 15846 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.447651 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:58.448493 15760 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.480782 15830 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.482077 15803 ts_tablet_manager.cc:1434] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.163s	user 0.010s	sys 0.007s
I20260709 13:16:58.483114 15803 tablet_bootstrap.cc:492] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058: Bootstrap starting.
I20260709 13:16:58.491653 15651 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058
I20260709 13:16:58.492671 15842 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.494360 15686 raft_consensus.cc:2393] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ebaa234beb6544fc9c9151e623044058 in current term 1: Already voted for candidate 0ad934ae28874ff083d1c1305fc718d1 in this term.
I20260709 13:16:58.498466 15804 tablet_bootstrap.cc:654] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.500207 15649 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058
I20260709 13:16:58.500922 15843 raft_consensus.cc:2749] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:58.502377 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.503783 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.485489 15839 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.506443 15608 raft_consensus.cc:2368] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 for earlier term 1. Current term is 2.
I20260709 13:16:58.506413 15724 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:58.507481 15758 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:58.508211 15758 raft_consensus.cc:2393] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 668cfc1d74d2440cacb6b0a6afb2c50e in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:58.509053 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:58.509966 15839 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.515472 15649 leader_election.cc:400] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Vote denied by peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219) with higher term. Message: Invalid argument: T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 for earlier term 1. Current term is 2.
I20260709 13:16:58.515790 15649 leader_election.cc:403] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Cancelling election due to peer responding with higher term
I20260709 13:16:58.515692 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.509819 15759 raft_consensus.cc:2393] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:58.516274 15686 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:58.510195 15834 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.500720 15846 raft_consensus.cc:515] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.505524 15830 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.516624 15845 raft_consensus.cc:515] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.520689 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.523082 15848 raft_consensus.cc:2804] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.523576 15848 raft_consensus.cc:697] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:58.513689 15760 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 1.
I20260709 13:16:58.524662 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.524528 15845 leader_election.cc:290] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.525802 15842 raft_consensus.cc:2749] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:58.510900 15723 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 0ad934ae28874ff083d1c1305fc718d1
W20260709 13:16:58.528432 15575 leader_election.cc:343] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.524384 15848 consensus_queue.cc:237] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.530076 15834 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.530736 15834 raft_consensus.cc:697] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:58.531836 15834 consensus_queue.cc:237] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.536290 15686 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
W20260709 13:16:58.529713 15723 leader_election.cc:343] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.536803 15843 raft_consensus.cc:2749] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election lost for term 1. Reason: Vote denied by peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219) with higher term. Message: Invalid argument: T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 for earlier term 1. Current term is 2.
I20260709 13:16:58.541209 15682 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 1.
I20260709 13:16:58.541481 15805 tablet_bootstrap.cc:492] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:58.533311 15846 leader_election.cc:290] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.542534 15575 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.543817 15845 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:58.544275 15845 raft_consensus.cc:697] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:58.545549 15805 ts_tablet_manager.cc:1403] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.232s	user 0.013s	sys 0.048s
I20260709 13:16:58.545725 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.539157 15505 catalog_manager.cc:5697] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.545835 15845 consensus_queue.cc:237] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.550602 15576 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058
I20260709 13:16:58.551618 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.552181 15608 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.552228 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:58.552896 15759 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.553025 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:58.548332 15724 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:58.550840 15686 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.556988 15839 raft_consensus.cc:2749] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:16:58.560235 15804 tablet_bootstrap.cc:492] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:58.561292 15804 ts_tablet_manager.cc:1403] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.201s	user 0.004s	sys 0.013s
W20260709 13:16:58.561535 15556 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:16:58.562431 15848 raft_consensus.cc:2804] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.563472 15848 raft_consensus.cc:697] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:58.564325 15848 consensus_queue.cc:237] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.563985 15505 catalog_manager.cc:5697] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.549300 15805 raft_consensus.cc:359] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.570770 15805 raft_consensus.cc:385] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.571007 15846 raft_consensus.cc:493] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.571218 15805 raft_consensus.cc:740] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.571553 15846 raft_consensus.cc:515] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.592221 15803 tablet_bootstrap.cc:654] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.570603 15804 raft_consensus.cc:359] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.572065 15805 consensus_queue.cc:260] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.668602 15804 raft_consensus.cc:385] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.668948 15804 raft_consensus.cc:740] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.669811 15686 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 1.
I20260709 13:16:58.670396 15805 ts_tablet_manager.cc:1434] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.124s	user 0.004s	sys 0.003s
I20260709 13:16:58.669834 15804 consensus_queue.cc:260] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.668344 15506 catalog_manager.cc:5697] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 0 to 2, leader changed from <none> to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.671810 15724 leader_election.cc:304] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:58.671823 15805 tablet_bootstrap.cc:492] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:58.673077 15845 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.673569 15845 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.674780 15759 raft_consensus.cc:2468] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 1.
I20260709 13:16:58.676712 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.677096 15576 leader_election.cc:304] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:58.677558 15686 raft_consensus.cc:2393] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 668cfc1d74d2440cacb6b0a6afb2c50e in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:58.678213 15839 raft_consensus.cc:2804] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.679533 15848 raft_consensus.cc:2804] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.679953 15839 raft_consensus.cc:697] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:58.680105 15848 raft_consensus.cc:697] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:58.680768 15848 consensus_queue.cc:237] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.680819 15839 consensus_queue.cc:237] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.678839 15845 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.684736 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.685273 15804 ts_tablet_manager.cc:1434] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.124s	user 0.021s	sys 0.017s
I20260709 13:16:58.686691 15804 tablet_bootstrap.cc:492] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:58.686997 15576 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058
I20260709 13:16:58.688118 15845 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.595180 15608 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 1.
I20260709 13:16:58.693717 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1c6bccba264b9b9301ef8b1bde7e8e" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
W20260709 13:16:58.695674 15723 leader_election.cc:343] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.678817 15805 tablet_bootstrap.cc:654] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.705915 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1c6bccba264b9b9301ef8b1bde7e8e" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.707181 15686 raft_consensus.cc:2468] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 0.
I20260709 13:16:58.698822 15845 raft_consensus.cc:2749] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:58.714766 15724 leader_election.cc:304] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:58.718443 15834 raft_consensus.cc:2804] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.719502 15834 raft_consensus.cc:493] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.719832 15834 raft_consensus.cc:3060] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.723625 15803 tablet_bootstrap.cc:492] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058: No bootstrap required, opened a new log
I20260709 13:16:58.724032 15803 ts_tablet_manager.cc:1403] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058: Time spent bootstrapping tablet: real 0.241s	user 0.026s	sys 0.034s
I20260709 13:16:58.731892 15804 tablet_bootstrap.cc:654] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.732143 15834 raft_consensus.cc:515] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.718168 15846 leader_election.cc:290] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.737996 15834 leader_election.cc:290] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:58.736922 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1c6bccba264b9b9301ef8b1bde7e8e" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:58.739306 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1c6bccba264b9b9301ef8b1bde7e8e" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.739843 15686 raft_consensus.cc:3060] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:16:58.739976 15723 leader_election.cc:343] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:16:58.742519 15805 tablet_bootstrap.cc:492] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:58.743005 15805 ts_tablet_manager.cc:1403] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.072s	user 0.006s	sys 0.014s
I20260709 13:16:58.743302 15804 tablet_bootstrap.cc:492] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:58.743748 15804 ts_tablet_manager.cc:1403] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.057s	user 0.008s	sys 0.006s
I20260709 13:16:58.746099 15805 raft_consensus.cc:359] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.746763 15805 raft_consensus.cc:385] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.747066 15805 raft_consensus.cc:740] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.747260 15804 raft_consensus.cc:359] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.747961 15804 raft_consensus.cc:385] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.748296 15804 raft_consensus.cc:740] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.747860 15805 consensus_queue.cc:260] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.751562 15686 raft_consensus.cc:2468] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 1.
I20260709 13:16:58.752839 15724 leader_election.cc:304] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:58.753902 15834 raft_consensus.cc:2804] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.754397 15834 raft_consensus.cc:697] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:58.755291 15834 consensus_queue.cc:237] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.748987 15804 consensus_queue.cc:260] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.758792 15803 raft_consensus.cc:359] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.759665 15803 raft_consensus.cc:385] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.759963 15803 raft_consensus.cc:740] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.760545 15804 ts_tablet_manager.cc:1434] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:16:58.761164 15805 ts_tablet_manager.cc:1434] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:16:58.761690 15804 tablet_bootstrap.cc:492] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: Bootstrap starting.
I20260709 13:16:58.762068 15805 tablet_bootstrap.cc:492] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: Bootstrap starting.
I20260709 13:16:58.779033 15508 catalog_manager.cc:5697] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.780354 15508 catalog_manager.cc:5697] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.770959 15804 tablet_bootstrap.cc:654] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.787160 15803 consensus_queue.cc:260] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.789206 15803 ts_tablet_manager.cc:1434] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.000s
I20260709 13:16:58.792130 15805 tablet_bootstrap.cc:654] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: Neither blocks nor log segments found. Creating new log.
I20260709 13:16:58.795812 15804 tablet_bootstrap.cc:492] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: No bootstrap required, opened a new log
I20260709 13:16:58.796267 15804 ts_tablet_manager.cc:1403] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.004s
I20260709 13:16:58.794935 15505 catalog_manager.cc:5697] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 0 to 1, leader changed from <none> to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.800415 15804 raft_consensus.cc:359] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.801151 15804 raft_consensus.cc:385] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.801487 15804 raft_consensus.cc:740] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.802265 15804 consensus_queue.cc:260] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.804330 15804 ts_tablet_manager.cc:1434] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:16:58.809031 15805 tablet_bootstrap.cc:492] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: No bootstrap required, opened a new log
I20260709 13:16:58.809571 15805 ts_tablet_manager.cc:1403] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.008s
I20260709 13:16:58.812495 15805 raft_consensus.cc:359] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.813225 15805 raft_consensus.cc:385] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:16:58.812367 15508 catalog_manager.cc:5697] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.813534 15805 raft_consensus.cc:740] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Initialized, Role: FOLLOWER
I20260709 13:16:58.814814 15805 consensus_queue.cc:260] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.814569 15508 catalog_manager.cc:5697] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:58.817271 15805 ts_tablet_manager.cc:1434] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:16:58.831096 15843 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:58.840643 15843 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:58.867450 15839 raft_consensus.cc:493] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.867959 15839 raft_consensus.cc:515] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.870079 15839 leader_election.cc:290] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.870944 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:58.871665 15686 raft_consensus.cc:2468] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.871645 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:58.872958 15575 leader_election.cc:304] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.873052 15759 raft_consensus.cc:2468] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 0.
I20260709 13:16:58.873690 15760 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.873842 15839 raft_consensus.cc:2804] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:16:58.874269 15839 raft_consensus.cc:493] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.874606 15839 raft_consensus.cc:3060] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.880666 15843 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:58.885214 15839 raft_consensus.cc:515] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.888353 15686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:58.888988 15686 raft_consensus.cc:3060] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.889122 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:58.889928 15759 raft_consensus.cc:3060] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:58.895296 15839 leader_election.cc:290] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:58.895798 15811 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:16:58.904191 15686 raft_consensus.cc:2468] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 1.
I20260709 13:16:58.907928 15575 leader_election.cc:304] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:58.909155 15839 raft_consensus.cc:2804] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:16:58.909777 15839 raft_consensus.cc:697] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:58.910667 15839 consensus_queue.cc:237] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:58.913897 15759 raft_consensus.cc:2468] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 1.
I20260709 13:16:58.941298 15507 catalog_manager.cc:5697] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 0 to 1, leader changed from <none> to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:58.943230 15834 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:58.958647 15833 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:58.986606 15833 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.007539 15848 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.020051 15682 raft_consensus.cc:1275] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:16:59.020380 15685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:54822
I20260709 13:16:59.021082 15685 raft_consensus.cc:606] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Received request to transfer leadership to TS 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:59.021800 15839 consensus_queue.cc:1048] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.028970 15682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:54822
I20260709 13:16:59.029613 15682 raft_consensus.cc:606] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Received request to transfer leadership to TS 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:59.044879 15686 raft_consensus.cc:1240] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Rejecting Update request from peer ebaa234beb6544fc9c9151e623044058 for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:59.047606 15759 raft_consensus.cc:3055] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.048048 15759 raft_consensus.cc:740] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:59.048913 15759 consensus_queue.cc:260] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.050191 15759 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.052211 15845 raft_consensus.cc:493] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.052693 15845 raft_consensus.cc:515] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.055857 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1c6bccba264b9b9301ef8b1bde7e8e" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ad934ae28874ff083d1c1305fc718d1" is_pre_election: true
I20260709 13:16:59.056233 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1c6bccba264b9b9301ef8b1bde7e8e" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:59.056687 15685 raft_consensus.cc:2393] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 668cfc1d74d2440cacb6b0a6afb2c50e in current term 1: Already voted for candidate ebaa234beb6544fc9c9151e623044058 in this term.
I20260709 13:16:59.054740 15845 leader_election.cc:290] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.058588 15575 leader_election.cc:304] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058
I20260709 13:16:59.059837 15845 raft_consensus.cc:3060] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:59.066078 15759 raft_consensus.cc:1275] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:16:59.066814 15759 pending_rounds.cc:85] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: Aborting all ops after (but not including) 0
I20260709 13:16:59.067225 15759 pending_rounds.cc:107] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:16:59.067579 15759 raft_consensus.cc:2889] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:59.068879 15845 raft_consensus.cc:2749] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:59.069449 15839 consensus_queue.cc:1048] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.070775 15760 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
mode: GRACEFUL
new_leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:53508
I20260709 13:16:59.071457 15760 raft_consensus.cc:606] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Received request to transfer leadership to TS 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:59.078370 15759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:53508
I20260709 13:16:59.078967 15759 raft_consensus.cc:606] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:59.084385 15759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:53508
I20260709 13:16:59.084895 15759 raft_consensus.cc:606] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:59.098914 15609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:44206
I20260709 13:16:59.099597 15609 raft_consensus.cc:606] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:59.105510 15609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:44206
I20260709 13:16:59.106151 15609 raft_consensus.cc:606] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:16:59.107437 15833 consensus_queue.cc:1048] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.108644 15843 raft_consensus.cc:493] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.109102 15843 raft_consensus.cc:515] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.110994 15843 leader_election.cc:290] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.113819 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebaa234beb6544fc9c9151e623044058" is_pre_election: true
I20260709 13:16:59.114579 15759 raft_consensus.cc:2393] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0ad934ae28874ff083d1c1305fc718d1 in current term 1: Already voted for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in this term.
I20260709 13:16:59.114720 15685 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:16:59.118495 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" is_pre_election: true
I20260709 13:16:59.120828 15649 leader_election.cc:304] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058
I20260709 13:16:59.122059 15843 raft_consensus.cc:2749] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:16:59.123003 15839 consensus_queue.cc:1048] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.124049 15833 consensus_queue.cc:1048] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:59.133114 15848 raft_consensus.cc:993] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.133704 15848 raft_consensus.cc:1081] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.139101 15685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54806
I20260709 13:16:59.140298 15685 raft_consensus.cc:493] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.140736 15685 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.147693 15685 raft_consensus.cc:515] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.150909 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:59.156441 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.157085 15759 raft_consensus.cc:3055] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.157449 15759 raft_consensus.cc:740] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:59.159981 15759 consensus_queue.cc:260] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.160580 15839 raft_consensus.cc:993] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.161212 15685 leader_election.cc:290] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.163890 15759 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.164136 15810 raft_consensus.cc:1081] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.165815 15685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54804
I20260709 13:16:59.166493 15685 raft_consensus.cc:493] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.166842 15685 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.174171 15759 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.175031 15685 raft_consensus.cc:515] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.178663 15651 leader_election.cc:304] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:16:59.179615 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.179759 15843 raft_consensus.cc:2804] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.179944 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:59.180215 15760 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.180311 15843 raft_consensus.cc:697] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:59.180886 15682 raft_consensus.cc:1240] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Rejecting Update request from peer 668cfc1d74d2440cacb6b0a6afb2c50e for earlier term 1. Current term is 2. Ops: []
I20260709 13:16:59.181221 15843 consensus_queue.cc:237] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.183133 15810 consensus_queue.cc:1048] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.185072 15685 leader_election.cc:290] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.187812 15833 consensus_queue.cc:1048] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.188336 15810 consensus_queue.cc:1059] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.191190 15608 raft_consensus.cc:3055] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.191561 15608 raft_consensus.cc:740] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:59.192358 15608 consensus_queue.cc:260] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.193168 15760 raft_consensus.cc:2468] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.193509 15608 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.194075 15759 raft_consensus.cc:1240] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Rejecting Update request from peer 668cfc1d74d2440cacb6b0a6afb2c50e for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:16:59.198027 15651 leader_election.cc:304] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:59.198963 15843 raft_consensus.cc:2804] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.199478 15843 raft_consensus.cc:697] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:59.200311 15843 consensus_queue.cc:237] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.203874 15608 raft_consensus.cc:2468] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.212657 15846 consensus_queue.cc:1048] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.212976 15507 catalog_manager.cc:5697] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 1 to 2, leader changed from ebaa234beb6544fc9c9151e623044058 (127.15.29.3) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.214746 15507 catalog_manager.cc:5697] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 1 to 2, leader changed from 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.256112 15830 raft_consensus.cc:993] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:16:59.256629 15830 raft_consensus.cc:1081] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Signalling peer 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:16:59.259079 15608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:16:59.259806 15608 raft_consensus.cc:493] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.260268 15608 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.265359 15608 raft_consensus.cc:515] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:59.267236 15608 leader_election.cc:290] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Requested vote from peers ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:16:59.268011 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.268488 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:59.268625 15760 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.269089 15685 raft_consensus.cc:3055] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.269469 15685 raft_consensus.cc:740] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:59.270339 15685 consensus_queue.cc:260] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:59.271665 15685 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.275626 15760 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
I20260709 13:16:59.276870 15576 leader_election.cc:304] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:59.277705 15839 raft_consensus.cc:2804] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.278187 15839 raft_consensus.cc:697] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:59.278191 15685 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
I20260709 13:16:59.279186 15839 consensus_queue.cc:237] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:16:59.285981 15508 catalog_manager.cc:5697] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 1 to 2, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.472481 15830 raft_consensus.cc:993] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:16:59.472931 15843 raft_consensus.cc:1081] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Signalling peer 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:16:59.474370 15608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:16:59.474946 15608 raft_consensus.cc:493] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.475314 15608 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.481225 15846 raft_consensus.cc:993] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.481689 15846 raft_consensus.cc:1081] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.483114 15608 raft_consensus.cc:515] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.483652 15685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54806
I20260709 13:16:59.484345 15685 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.484762 15685 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.485699 15608 leader_election.cc:290] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.486275 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:59.486856 15682 raft_consensus.cc:3055] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.487201 15682 raft_consensus.cc:740] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:59.487340 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.487941 15760 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.487995 15682 consensus_queue.cc:260] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.489307 15682 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.492687 15685 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.494596 15760 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
I20260709 13:16:59.495420 15685 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.495868 15576 leader_election.cc:304] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:59.496106 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.496757 15839 raft_consensus.cc:2804] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.497251 15839 raft_consensus.cc:697] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:59.496701 15760 raft_consensus.cc:3055] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.498550 15760 raft_consensus.cc:740] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:59.499465 15760 consensus_queue.cc:260] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.499924 15608 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:59.500499 15608 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.500727 15760 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.500460 15810 consensus_queue.cc:1048] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:16:59.498072 15839 consensus_queue.cc:237] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.508230 15760 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.510408 15608 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.513882 15651 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:16:59.514736 15843 raft_consensus.cc:2804] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.515357 15843 raft_consensus.cc:697] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:59.516170 15843 consensus_queue.cc:237] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.521169 15682 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
I20260709 13:16:59.523118 15508 catalog_manager.cc:5697] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 1 to 2, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.528019 15839 consensus_queue.cc:1048] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.540320 15507 catalog_manager.cc:5697] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 1 to 2, leader changed from ebaa234beb6544fc9c9151e623044058 (127.15.29.3) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.551394 15810 raft_consensus.cc:993] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.553779 15839 raft_consensus.cc:1081] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:16:59.555604 15682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54804
I20260709 13:16:59.556219 15682 raft_consensus.cc:493] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.556607 15682 raft_consensus.cc:3060] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.561555 15833 raft_consensus.cc:993] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: : Instructing follower 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:16:59.562085 15833 raft_consensus.cc:1081] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Signalling peer 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:16:59.563556 15682 raft_consensus.cc:515] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.565834 15682 leader_election.cc:290] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.567813 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.568477 15759 raft_consensus.cc:3060] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.569083 15608 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:44198
I20260709 13:16:59.569720 15608 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:59.570149 15608 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.572435 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089885ab0d8541f5a1853955cc74c166" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:16:59.573084 15609 raft_consensus.cc:3055] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.573417 15609 raft_consensus.cc:740] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:59.574239 15609 consensus_queue.cc:260] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.575714 15609 raft_consensus.cc:3060] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.578634 15608 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.582823 15609 raft_consensus.cc:2468] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.584703 15649 leader_election.cc:304] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:59.585359 15611 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.585616 15830 raft_consensus.cc:2804] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.586144 15830 raft_consensus.cc:697] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:16:59.587014 15830 consensus_queue.cc:237] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.588258 15608 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:16:59.587707 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:16:59.589592 15682 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.590313 15610 raft_consensus.cc:1275] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.592607 15760 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:16:59.593181 15760 raft_consensus.cc:3055] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:16:59.593492 15760 raft_consensus.cc:740] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:16:59.594257 15760 consensus_queue.cc:260] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.596010 15760 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:16:59.596580 15682 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
I20260709 13:16:59.596645 15507 catalog_manager.cc:5697] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 1 to 2, leader changed from 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.600298 15575 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:16:59.601245 15839 raft_consensus.cc:2804] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:16:59.601744 15839 raft_consensus.cc:697] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:16:59.601732 15830 consensus_queue.cc:1048] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:16:59.602748 15839 consensus_queue.cc:237] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:16:59.607735 15760 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 2.
I20260709 13:16:59.610028 15611 raft_consensus.cc:1275] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.613063 15843 consensus_queue.cc:1048] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.606338 15759 raft_consensus.cc:2468] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 2.
I20260709 13:16:59.615948 15508 catalog_manager.cc:5697] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 1 to 2, leader changed from ebaa234beb6544fc9c9151e623044058 (127.15.29.3) to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:59.623956 15759 raft_consensus.cc:1275] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.625473 15830 consensus_queue.cc:1048] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:16:59.642484 15759 raft_consensus.cc:1275] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.644049 15893 consensus_queue.cc:1048] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.798103 15682 raft_consensus.cc:1275] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.799520 15839 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.810103 15759 raft_consensus.cc:1275] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.812224 15845 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.947557 15760 raft_consensus.cc:1275] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.948910 15845 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:16:59.957526 15685 raft_consensus.cc:1275] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:16:59.959337 15839 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.009686 15609 raft_consensus.cc:1275] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.011241 15893 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.013191 15760 raft_consensus.cc:1275] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.014662 15839 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:00.023387 15759 raft_consensus.cc:1275] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.026190 15830 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.036161 15685 raft_consensus.cc:1275] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.037498 15810 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.068338 15759 raft_consensus.cc:1275] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.069463 15902 consensus_queue.cc:1048] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:00.077221 15609 raft_consensus.cc:1275] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:00.078790 15843 consensus_queue.cc:1048] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:02.141500 15476 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:17:02.142266 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:02.158038 15610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
mode: GRACEFUL
new_leader_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:44222
I20260709 13:17:02.158583 15610 raft_consensus.cc:606] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Received request to transfer leadership to TS ebaa234beb6544fc9c9151e623044058
I20260709 13:17:02.169339 15685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:54834
I20260709 13:17:02.169859 15685 raft_consensus.cc:606] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Received request to transfer leadership to TS ebaa234beb6544fc9c9151e623044058
I20260709 13:17:02.171054 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:02.171481 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:02.337229 15810 raft_consensus.cc:993] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: : Instructing follower ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:02.337689 15908 raft_consensus.cc:1081] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Signalling peer ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:02.339133 15759 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:53500
I20260709 13:17:02.339607 15759 raft_consensus.cc:493] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.339912 15759 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.344463 15759 raft_consensus.cc:515] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:02.346241 15759 leader_election.cc:290] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 3 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:17:02.347138 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:02.347381 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:02.347705 15610 raft_consensus.cc:3055] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.347946 15685 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.348063 15610 raft_consensus.cc:740] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:02.348917 15610 consensus_queue.cc:260] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:02.349869 15610 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.353469 15685 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 3.
I20260709 13:17:02.354621 15724 leader_election.cc:304] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:02.354748 15610 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 3.
I20260709 13:17:02.355432 15848 raft_consensus.cc:2804] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:02.356472 15848 raft_consensus.cc:697] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:17:02.357257 15848 consensus_queue.cc:237] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:02.365679 15505 catalog_manager.cc:5697] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 2 to 3, leader changed from 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1) to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 3 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.600351 15903 raft_consensus.cc:993] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:02.600812 15903 raft_consensus.cc:1081] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Signalling peer ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:02.602393 15759 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:53486
I20260709 13:17:02.602993 15759 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:02.603318 15759 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.609046 15759 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:02.611153 15759 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 3 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:17:02.611496 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:02.611720 15685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:02.612180 15610 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.612326 15685 raft_consensus.cc:3055] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:02.612694 15685 raft_consensus.cc:740] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:02.613318 15685 consensus_queue.cc:260] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:02.614127 15685 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:02.618651 15685 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 3.
I20260709 13:17:02.619463 15610 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 3.
I20260709 13:17:02.619796 15724 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:02.620576 15848 raft_consensus.cc:2804] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:02.621013 15848 raft_consensus.cc:697] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:17:02.621778 15848 consensus_queue.cc:237] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:02.628350 15505 catalog_manager.cc:5697] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 2 to 3, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 3 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.778012 15685 raft_consensus.cc:1275] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:02.779353 15848 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:02.782835 15610 raft_consensus.cc:1275] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:02.784346 15848 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:03.006271 15610 raft_consensus.cc:1275] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.007674 15848 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:03.016327 15685 raft_consensus.cc:1275] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.017930 15909 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:03.172191 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:03.182287 15610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:44206
I20260709 13:17:03.182963 15610 raft_consensus.cc:606] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:17:03.187714 15759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:53508
I20260709 13:17:03.188256 15759 raft_consensus.cc:606] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:17:03.192883 15610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:44206
I20260709 13:17:03.193388 15610 raft_consensus.cc:606] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:17:03.197633 15610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:44206
I20260709 13:17:03.198160 15610 raft_consensus.cc:606] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:17:03.204918 15759 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
mode: GRACEFUL
new_leader_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:53508
I20260709 13:17:03.205473 15759 raft_consensus.cc:606] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Received request to transfer leadership to TS 0ad934ae28874ff083d1c1305fc718d1
I20260709 13:17:03.414290 15918 raft_consensus.cc:993] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.414745 15909 raft_consensus.cc:1081] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.416180 15685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54806
I20260709 13:17:03.416703 15685 raft_consensus.cc:493] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.417055 15685 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.421932 15685 raft_consensus.cc:515] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.423652 15685 leader_election.cc:290] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 4 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:03.424660 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:03.424672 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dda50be1bbe4c698a0bab707d89017a" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:03.425419 15759 raft_consensus.cc:3055] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:03.425459 15610 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.425769 15759 raft_consensus.cc:740] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:17:03.426584 15759 consensus_queue.cc:260] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.427644 15759 raft_consensus.cc:3060] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.431169 15610 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 4.
I20260709 13:17:03.432024 15759 raft_consensus.cc:2468] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 4.
I20260709 13:17:03.432425 15649 leader_election.cc:304] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:17:03.433234 15903 raft_consensus.cc:2804] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:03.434096 15903 raft_consensus.cc:697] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 4 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:03.434857 15903 consensus_queue.cc:237] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.440739 15926 raft_consensus.cc:993] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.441246 15908 raft_consensus.cc:1081] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.442795 15685 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54804
I20260709 13:17:03.442368 15507 catalog_manager.cc:5697] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 3 to 4, leader changed from ebaa234beb6544fc9c9151e623044058 (127.15.29.3) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.443310 15685 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.443665 15685 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.448997 15685 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.450800 15926 raft_consensus.cc:993] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.451320 15926 raft_consensus.cc:1081] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.452090 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:03.452461 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:03.452638 15759 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.453004 15610 raft_consensus.cc:3055] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.453177 15682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54804
I20260709 13:17:03.453365 15610 raft_consensus.cc:740] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:03.453773 15682 raft_consensus.cc:493] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.454166 15682 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.454149 15610 consensus_queue.cc:260] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.455399 15610 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.458951 15759 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 3.
I20260709 13:17:03.459023 15685 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 3 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:03.460394 15651 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:03.460731 15682 raft_consensus.cc:515] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:17:03.461256 15903 raft_consensus.cc:2804] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:03.461736 15903 raft_consensus.cc:697] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:03.462083 15610 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 3.
I20260709 13:17:03.462571 15903 consensus_queue.cc:237] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.463812 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:03.464385 15682 leader_election.cc:290] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 3 election: Requested vote from peers ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095), 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219)
I20260709 13:17:03.464394 15759 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.464486 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:03.466018 15610 raft_consensus.cc:3055] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.466485 15610 raft_consensus.cc:740] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:03.467377 15610 consensus_queue.cc:260] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:17:03.468581 15610 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.470532 15507 catalog_manager.cc:5697] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 2 to 3, leader changed from 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.473584 15759 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 3.
I20260709 13:17:03.474946 15651 leader_election.cc:304] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:03.475064 15610 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 3.
I20260709 13:17:03.475863 15903 raft_consensus.cc:2804] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:03.476444 15903 raft_consensus.cc:697] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:03.477289 15903 consensus_queue.cc:237] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:17:03.484337 15507 catalog_manager.cc:5697] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 2 to 3, leader changed from 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.555619 15918 raft_consensus.cc:993] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.556106 15918 raft_consensus.cc:1081] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.557891 15682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54806
I20260709 13:17:03.558483 15682 raft_consensus.cc:493] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.558876 15682 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.563930 15682 raft_consensus.cc:515] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.565708 15682 leader_election.cc:290] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 4 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:03.566490 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:03.566779 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea0971e8177a4bf99eb967030baf8483" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:03.567128 15610 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.567337 15759 raft_consensus.cc:3055] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:03.567682 15759 raft_consensus.cc:740] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:17:03.568250 15759 consensus_queue.cc:260] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.569038 15759 raft_consensus.cc:3060] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:03.572113 15610 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 4.
I20260709 13:17:03.573153 15759 raft_consensus.cc:2468] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 4.
I20260709 13:17:03.573128 15649 leader_election.cc:304] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, 668cfc1d74d2440cacb6b0a6afb2c50e; no voters: 
I20260709 13:17:03.574056 15903 raft_consensus.cc:2804] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:03.574538 15903 raft_consensus.cc:697] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:03.575265 15903 consensus_queue.cc:237] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.580576 15507 catalog_manager.cc:5697] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 3 to 4, leader changed from ebaa234beb6544fc9c9151e623044058 (127.15.29.3) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.664242 15908 raft_consensus.cc:993] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: : Instructing follower 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.664716 15926 raft_consensus.cc:1081] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Signalling peer 0ad934ae28874ff083d1c1305fc718d1 to start an election
I20260709 13:17:03.666299 15682 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
 from {username='slave'} at 127.0.0.1:54804
I20260709 13:17:03.666778 15682 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:03.667124 15682 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.671626 15682 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.673265 15682 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 3 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:03.673961 15610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:03.674216 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "0ad934ae28874ff083d1c1305fc718d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:03.674549 15610 raft_consensus.cc:3055] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:03.674753 15759 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.674932 15610 raft_consensus.cc:740] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:03.675721 15610 consensus_queue.cc:260] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.676599 15610 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:03.678810 15759 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 3.
I20260709 13:17:03.679811 15651 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ad934ae28874ff083d1c1305fc718d1, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:03.680455 15903 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:03.680876 15903 raft_consensus.cc:697] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Becoming Leader. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:03.681373 15610 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ad934ae28874ff083d1c1305fc718d1 in term 3.
I20260709 13:17:03.681596 15903 consensus_queue.cc:237] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:03.687774 15507 catalog_manager.cc:5697] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 reported cstate change: term changed from 2 to 3, leader changed from 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1) to 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "0ad934ae28874ff083d1c1305fc718d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.958225 15759 raft_consensus.cc:1275] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:03.959543 15903 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:03.960515 15610 raft_consensus.cc:1275] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:03.961823 15903 consensus_queue.cc:1048] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:03.976925 15760 raft_consensus.cc:1275] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.979449 15903 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:03.984838 15759 raft_consensus.cc:1275] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.987222 15938 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:03.994288 15610 raft_consensus.cc:1275] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:03.995934 15938 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:04.003393 15609 raft_consensus.cc:1275] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:04.004668 15925 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:04.053433 15759 raft_consensus.cc:1275] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:04.055092 15925 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:04.066728 15609 raft_consensus.cc:1275] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:04.068933 15942 consensus_queue.cc:1048] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:04.134438 15759 raft_consensus.cc:1275] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:04.135710 15942 consensus_queue.cc:1048] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:04.143630 15609 raft_consensus.cc:1275] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Refusing update from remote peer 0ad934ae28874ff083d1c1305fc718d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:04.145958 15942 consensus_queue.cc:1048] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:06.215170 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:06.222095 15682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:54834
I20260709 13:17:06.222786 15682 raft_consensus.cc:606] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Received request to transfer leadership to TS 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:17:06.225704 15682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:54834
I20260709 13:17:06.226135 15682 raft_consensus.cc:606] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Received request to transfer leadership to TS 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:17:06.228296 15682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:54834
I20260709 13:17:06.228713 15682 raft_consensus.cc:606] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Received request to transfer leadership to TS ebaa234beb6544fc9c9151e623044058
I20260709 13:17:06.230952 15682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:54834
I20260709 13:17:06.231338 15682 raft_consensus.cc:606] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Received request to transfer leadership to TS 668cfc1d74d2440cacb6b0a6afb2c50e
I20260709 13:17:06.233362 15682 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040"
dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
mode: GRACEFUL
new_leader_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:54834
I20260709 13:17:06.233745 15682 raft_consensus.cc:606] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Received request to transfer leadership to TS ebaa234beb6544fc9c9151e623044058
I20260709 13:17:06.234745 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:17:06.235229 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:06.318410 15938 raft_consensus.cc:993] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:06.318921 15938 raft_consensus.cc:1081] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Signalling peer ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:06.320621 15759 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:53486
I20260709 13:17:06.321089 15759 raft_consensus.cc:493] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.321409 15759 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.326437 15759 raft_consensus.cc:515] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.328030 15759 leader_election.cc:290] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 3 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:17:06.329151 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:06.329258 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6ca0bf372de4e199c8636d495a01040" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:06.329962 15682 raft_consensus.cc:3055] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:06.330065 15609 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.330230 15682 raft_consensus.cc:740] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:06.331142 15682 consensus_queue.cc:260] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.332211 15682 raft_consensus.cc:3060] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.335636 15609 raft_consensus.cc:2468] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 3.
I20260709 13:17:06.337266 15723 leader_election.cc:304] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:06.337837 15682 raft_consensus.cc:2468] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 3.
I20260709 13:17:06.338433 15950 raft_consensus.cc:2804] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:06.339824 15950 raft_consensus.cc:697] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:17:06.340740 15950 consensus_queue.cc:237] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.348763 15505 catalog_manager.cc:5697] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 2 to 3, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 3 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.389350 15951 raft_consensus.cc:993] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:06.389798 15938 raft_consensus.cc:1081] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Signalling peer ebaa234beb6544fc9c9151e623044058 to start an election
I20260709 13:17:06.391315 15759 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2"
dest_uuid: "ebaa234beb6544fc9c9151e623044058"
 from {username='slave'} at 127.0.0.1:53486
I20260709 13:17:06.391742 15759 raft_consensus.cc:493] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.391973 15759 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.395876 15759 raft_consensus.cc:515] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:17:06.397356 15759 leader_election.cc:290] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 4 election: Requested vote from peers 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219), 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253)
I20260709 13:17:06.398132 15609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
I20260709 13:17:06.398383 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "701ac14f84214ca6a9ff6c7be2291ef2" candidate_uuid: "ebaa234beb6544fc9c9151e623044058" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:06.398695 15609 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.399030 15682 raft_consensus.cc:3055] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:06.399356 15682 raft_consensus.cc:740] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:06.400156 15682 consensus_queue.cc:260] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:17:06.401062 15682 raft_consensus.cc:3060] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.403627 15609 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 4.
I20260709 13:17:06.404683 15723 leader_election.cc:304] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:06.405398 15950 raft_consensus.cc:2804] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:06.405867 15950 raft_consensus.cc:697] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 4 LEADER]: Becoming Leader. State: Replica: ebaa234beb6544fc9c9151e623044058, State: Running, Role: LEADER
I20260709 13:17:06.406213 15682 raft_consensus.cc:2468] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebaa234beb6544fc9c9151e623044058 in term 4.
I20260709 13:17:06.406919 15950 consensus_queue.cc:237] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } }
I20260709 13:17:06.414290 15505 catalog_manager.cc:5697] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 reported cstate change: term changed from 3 to 4, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to ebaa234beb6544fc9c9151e623044058 (127.15.29.3). New cstate: current_term: 4 leader_uuid: "ebaa234beb6544fc9c9151e623044058" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.595369 15951 raft_consensus.cc:993] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:17:06.595789 15938 raft_consensus.cc:1081] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Signalling peer 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:17:06.597128 15609 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:17:06.597548 15609 raft_consensus.cc:493] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.597801 15609 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.602246 15609 raft_consensus.cc:515] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.604010 15609 leader_election.cc:290] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 4 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:06.605052 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:06.605181 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c3ddd6a760b4ce18907522e6e096ff3" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:06.605639 15682 raft_consensus.cc:3055] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:06.605641 15759 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.606027 15682 raft_consensus.cc:740] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:06.606771 15682 consensus_queue.cc:260] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.607733 15682 raft_consensus.cc:3060] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.609954 15759 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 4.
I20260709 13:17:06.611213 15576 leader_election.cc:304] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:06.612788 15682 raft_consensus.cc:2468] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 4.
I20260709 13:17:06.612962 15960 raft_consensus.cc:2804] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:06.614228 15960 raft_consensus.cc:697] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:06.615074 15960 consensus_queue.cc:237] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.622063 15505 catalog_manager.cc:5697] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 3 to 4, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 4 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.628018 15951 raft_consensus.cc:993] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:17:06.628518 15938 raft_consensus.cc:1081] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Signalling peer 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:17:06.630117 15609 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:17:06.630601 15609 raft_consensus.cc:493] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.630954 15609 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.635375 15609 raft_consensus.cc:515] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.636854 15609 leader_election.cc:290] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 3 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:06.637687 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:06.637794 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9612e9531a6343ef9af8c5ca01d50c61" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:06.638314 15682 raft_consensus.cc:3055] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:06.638350 15759 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.638600 15682 raft_consensus.cc:740] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:06.639420 15682 consensus_queue.cc:260] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.640712 15682 raft_consensus.cc:3060] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:06.643733 15951 raft_consensus.cc:993] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: : Instructing follower 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:17:06.644184 15951 raft_consensus.cc:1081] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Signalling peer 668cfc1d74d2440cacb6b0a6afb2c50e to start an election
I20260709 13:17:06.644359 15759 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 3.
I20260709 13:17:06.645274 15682 raft_consensus.cc:2468] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 3.
I20260709 13:17:06.645764 15576 leader_election.cc:304] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:06.645895 15609 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548"
dest_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e"
 from {username='slave'} at 127.0.0.1:44182
I20260709 13:17:06.646513 15609 raft_consensus.cc:493] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:06.646737 15960 raft_consensus.cc:2804] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:06.646950 15609 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.647301 15960 raft_consensus.cc:697] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:06.648226 15960 consensus_queue.cc:237] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.654114 15609 raft_consensus.cc:515] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.656052 15609 leader_election.cc:290] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 4 election: Requested vote from peers 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2:35253), ebaa234beb6544fc9c9151e623044058 (127.15.29.3:46095)
I20260709 13:17:06.655967 15505 catalog_manager.cc:5697] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 2 to 3, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 3 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.659222 15682 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "0ad934ae28874ff083d1c1305fc718d1"
I20260709 13:17:06.659739 15682 raft_consensus.cc:3055] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:06.660009 15682 raft_consensus.cc:740] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0ad934ae28874ff083d1c1305fc718d1, State: Running, Role: LEADER
I20260709 13:17:06.659940 15759 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5b6fb9cb427a4e92a9931b6f98a88548" candidate_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "ebaa234beb6544fc9c9151e623044058"
I20260709 13:17:06.660600 15759 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.660789 15682 consensus_queue.cc:260] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.661947 15682 raft_consensus.cc:3060] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:06.666997 15759 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 4.
I20260709 13:17:06.667966 15576 leader_election.cc:304] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 668cfc1d74d2440cacb6b0a6afb2c50e, ebaa234beb6544fc9c9151e623044058; no voters: 
I20260709 13:17:06.668710 15960 raft_consensus.cc:2804] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:06.668870 15682 raft_consensus.cc:2468] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 668cfc1d74d2440cacb6b0a6afb2c50e in term 4.
I20260709 13:17:06.669173 15960 raft_consensus.cc:697] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 LEADER]: Becoming Leader. State: Replica: 668cfc1d74d2440cacb6b0a6afb2c50e, State: Running, Role: LEADER
I20260709 13:17:06.670112 15960 consensus_queue.cc:237] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } }
I20260709 13:17:06.682516 15505 catalog_manager.cc:5697] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e reported cstate change: term changed from 3 to 4, leader changed from 0ad934ae28874ff083d1c1305fc718d1 (127.15.29.2) to 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1). New cstate: current_term: 4 leader_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.869482 15682 raft_consensus.cc:1275] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:06.870725 15953 consensus_queue.cc:1048] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:06.882972 15609 raft_consensus.cc:1275] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:06.883598 15610 raft_consensus.cc:1275] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:06.885903 15953 consensus_queue.cc:1048] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:06.886087 15967 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "668cfc1d74d2440cacb6b0a6afb2c50e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 45219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:06.897502 15682 raft_consensus.cc:1275] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Refusing update from remote peer ebaa234beb6544fc9c9151e623044058: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:06.898914 15950 consensus_queue.cc:1048] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.120815 15759 raft_consensus.cc:1275] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:07.121989 15960 consensus_queue.cc:1048] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.130405 15682 raft_consensus.cc:1275] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:07.133478 15975 consensus_queue.cc:1048] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:07.163763 15682 raft_consensus.cc:1275] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:07.164897 15975 consensus_queue.cc:1048] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.173269 15759 raft_consensus.cc:1275] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:07.175160 15960 consensus_queue.cc:1048] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:07.192553 15759 raft_consensus.cc:1275] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:07.193708 15960 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebaa234beb6544fc9c9151e623044058" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 46095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.202555 15682 raft_consensus.cc:1275] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Refusing update from remote peer 668cfc1d74d2440cacb6b0a6afb2c50e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:07.203995 15960 consensus_queue.cc:1048] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ad934ae28874ff083d1c1305fc718d1" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:07.236099 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:07.241596 15476 tablet_server.cc:179] TabletServer@127.15.29.1:0 shutting down...
W20260709 13:17:07.246269 15723 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:07.250219 15723 consensus_peers.cc:597] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 -> Peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Couldn't send request to peer 668cfc1d74d2440cacb6b0a6afb2c50e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.271317 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.271970 15476 tablet_replica.cc:333] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.272675 15476 raft_consensus.cc:2243] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.273188 15476 raft_consensus.cc:2272] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 668cfc1d74d2440cacb6b0a6afb2c50e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.275584 15476 tablet_replica.cc:333] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.276106 15476 raft_consensus.cc:2243] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.276552 15476 raft_consensus.cc:2272] T ea0971e8177a4bf99eb967030baf8483 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.278203 15476 tablet_replica.cc:333] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.278687 15476 raft_consensus.cc:2243] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.279198 15476 raft_consensus.cc:2272] T 089885ab0d8541f5a1853955cc74c166 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.280889 15476 tablet_replica.cc:333] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.281386 15476 raft_consensus.cc:2243] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.281816 15476 raft_consensus.cc:2272] T 701ac14f84214ca6a9ff6c7be2291ef2 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.284049 15476 tablet_replica.cc:333] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.284569 15476 raft_consensus.cc:2243] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.285035 15476 raft_consensus.cc:2272] T b6ca0bf372de4e199c8636d495a01040 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.287133 15476 tablet_replica.cc:333] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.287665 15476 raft_consensus.cc:2243] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.288453 15476 raft_consensus.cc:2272] T 5b6fb9cb427a4e92a9931b6f98a88548 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.290503 15476 tablet_replica.cc:333] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.291146 15476 raft_consensus.cc:2243] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.291968 15476 raft_consensus.cc:2272] T 9612e9531a6343ef9af8c5ca01d50c61 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.294473 15476 tablet_replica.cc:333] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.294952 15476 raft_consensus.cc:2243] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.295320 15476 raft_consensus.cc:2272] T 8dda50be1bbe4c698a0bab707d89017a P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.296921 15476 tablet_replica.cc:333] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e: stopping tablet replica
I20260709 13:17:07.297310 15476 raft_consensus.cc:2243] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.297941 15476 raft_consensus.cc:2272] T 0c3ddd6a760b4ce18907522e6e096ff3 P 668cfc1d74d2440cacb6b0a6afb2c50e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.402235 15476 tablet_server.cc:196] TabletServer@127.15.29.1:0 shutdown complete.
W20260709 13:17:07.403381 15723 consensus_peers.cc:597] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 -> Peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Couldn't send request to peer 668cfc1d74d2440cacb6b0a6afb2c50e. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:45219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.405175 15649 consensus_peers.cc:597] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 -> Peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Couldn't send request to peer 668cfc1d74d2440cacb6b0a6afb2c50e. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:45219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.405560 15649 consensus_peers.cc:597] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 -> Peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Couldn't send request to peer 668cfc1d74d2440cacb6b0a6afb2c50e. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:45219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.405862 15649 consensus_peers.cc:597] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 -> Peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Couldn't send request to peer 668cfc1d74d2440cacb6b0a6afb2c50e. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:45219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.420779 15476 tablet_server.cc:179] TabletServer@127.15.29.2:0 shutting down...
I20260709 13:17:07.451659 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.452447 15476 tablet_replica.cc:333] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.453042 15476 raft_consensus.cc:2243] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.453493 15476 raft_consensus.cc:2272] T b6ca0bf372de4e199c8636d495a01040 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.455643 15476 tablet_replica.cc:333] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.456192 15476 raft_consensus.cc:2243] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.456660 15476 raft_consensus.cc:2272] T 0d1c6bccba264b9b9301ef8b1bde7e8e P 0ad934ae28874ff083d1c1305fc718d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.458345 15476 tablet_replica.cc:333] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.458835 15476 raft_consensus.cc:2243] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.459641 15476 raft_consensus.cc:2272] T ea0971e8177a4bf99eb967030baf8483 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.461263 15476 tablet_replica.cc:333] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.461786 15476 raft_consensus.cc:2243] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.462186 15476 raft_consensus.cc:2272] T 5b6fb9cb427a4e92a9931b6f98a88548 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.464572 15476 tablet_replica.cc:333] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.465041 15476 raft_consensus.cc:2243] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.465765 15476 raft_consensus.cc:2272] T 089885ab0d8541f5a1853955cc74c166 P 0ad934ae28874ff083d1c1305fc718d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.467501 15476 tablet_replica.cc:333] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.467988 15476 raft_consensus.cc:2243] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.468382 15476 raft_consensus.cc:2272] T 701ac14f84214ca6a9ff6c7be2291ef2 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.470400 15476 tablet_replica.cc:333] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.470929 15476 raft_consensus.cc:2243] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.471359 15476 raft_consensus.cc:2272] T 0c3ddd6a760b4ce18907522e6e096ff3 P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.473284 15476 tablet_replica.cc:333] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.473779 15476 raft_consensus.cc:2243] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.474172 15476 raft_consensus.cc:2272] T 9612e9531a6343ef9af8c5ca01d50c61 P 0ad934ae28874ff083d1c1305fc718d1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.476140 15476 tablet_replica.cc:333] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1: stopping tablet replica
I20260709 13:17:07.476601 15476 raft_consensus.cc:2243] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.477360 15476 raft_consensus.cc:2272] T 8dda50be1bbe4c698a0bab707d89017a P 0ad934ae28874ff083d1c1305fc718d1 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:07.493242 15723 consensus_peers.cc:597] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 -> Peer 668cfc1d74d2440cacb6b0a6afb2c50e (127.15.29.1:45219): Couldn't send request to peer 668cfc1d74d2440cacb6b0a6afb2c50e. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:45219: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.508221 15476 tablet_server.cc:196] TabletServer@127.15.29.2:0 shutdown complete.
I20260709 13:17:07.525627 15476 tablet_server.cc:179] TabletServer@127.15.29.3:0 shutting down...
I20260709 13:17:07.548230 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:07.548919 15476 tablet_replica.cc:333] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.549633 15476 raft_consensus.cc:2243] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.550347 15476 raft_consensus.cc:2272] T 0d1c6bccba264b9b9301ef8b1bde7e8e P ebaa234beb6544fc9c9151e623044058 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.552151 15476 tablet_replica.cc:333] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.552667 15476 raft_consensus.cc:2243] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.553411 15476 raft_consensus.cc:2272] T b6ca0bf372de4e199c8636d495a01040 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.555369 15476 tablet_replica.cc:333] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.555891 15476 raft_consensus.cc:2243] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.556339 15476 raft_consensus.cc:2272] T ea0971e8177a4bf99eb967030baf8483 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.558061 15476 tablet_replica.cc:333] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.558573 15476 raft_consensus.cc:2243] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.559089 15476 raft_consensus.cc:2272] T 0c3ddd6a760b4ce18907522e6e096ff3 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.561681 15476 tablet_replica.cc:333] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.562237 15476 raft_consensus.cc:2243] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.562698 15476 raft_consensus.cc:2272] T 9612e9531a6343ef9af8c5ca01d50c61 P ebaa234beb6544fc9c9151e623044058 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.564814 15476 tablet_replica.cc:333] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.565374 15476 raft_consensus.cc:2243] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.565865 15476 raft_consensus.cc:2272] T 8dda50be1bbe4c698a0bab707d89017a P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.567765 15476 tablet_replica.cc:333] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.568265 15476 raft_consensus.cc:2243] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.568694 15476 raft_consensus.cc:2272] T 5b6fb9cb427a4e92a9931b6f98a88548 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.570642 15476 tablet_replica.cc:333] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.571185 15476 raft_consensus.cc:2243] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.571631 15476 raft_consensus.cc:2272] T 089885ab0d8541f5a1853955cc74c166 P ebaa234beb6544fc9c9151e623044058 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.573230 15476 tablet_replica.cc:333] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058: stopping tablet replica
I20260709 13:17:07.573690 15476 raft_consensus.cc:2243] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.574344 15476 raft_consensus.cc:2272] T 701ac14f84214ca6a9ff6c7be2291ef2 P ebaa234beb6544fc9c9151e623044058 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.601533 15476 tablet_server.cc:196] TabletServer@127.15.29.3:0 shutdown complete.
I20260709 13:17:07.618321 15476 master.cc:562] Master@127.15.29.62:40789 shutting down...
I20260709 13:17:07.779687 15563 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:17:07.836942 15476 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:07.837498 15476 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:07.837795 15476 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9e758ef2a0cd4885965a7bf07143b8ab: stopping tablet replica
I20260709 13:17:07.857208 15476 master.cc:584] Master@127.15.29.62:40789 shutdown complete.
0ad934ae28874ff083d1c1305fc718d1  4
668cfc1d74d2440cacb6b0a6afb2c50e  4
ebaa234beb6544fc9c9151e623044058  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12840 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:17:07.898617 15476 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.29.62:33389
I20260709 13:17:07.899806 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:07.905607 15986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.905648 15985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:07.905865 15988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:07.906776 15476 server_base.cc:1061] running on GCE node
I20260709 13:17:07.907719 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:07.907928 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:07.908097 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603027908081 us; error 0 us; skew 500 ppm
I20260709 13:17:07.908623 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:07.911051 15476 webserver.cc:533] Webserver started at http://127.15.29.62:38757/ using document root <none> and password file <none>
I20260709 13:17:07.911585 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:07.911823 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:07.912184 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:07.913681 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/master-0-root/instance:
uuid: "572bfa6ecd224de29c4d5ea51babff36"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-hz8m"
I20260709 13:17:07.918668 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 13:17:07.921844 15993 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:07.922573 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:07.922847 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/master-0-root
uuid: "572bfa6ecd224de29c4d5ea51babff36"
format_stamp: "Formatted at 2026-07-09 13:17:07 on dist-test-slave-hz8m"
I20260709 13:17:07.923193 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:07.940110 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:07.941378 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:07.981192 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.62:33389
I20260709 13:17:07.981297 16044 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.62:33389 every 8 connection(s)
I20260709 13:17:07.985318 16045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:07.996157 16045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36: Bootstrap starting.
I20260709 13:17:08.001744 16045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.006464 16045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36: No bootstrap required, opened a new log
I20260709 13:17:08.008692 16045 raft_consensus.cc:359] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572bfa6ecd224de29c4d5ea51babff36" member_type: VOTER }
I20260709 13:17:08.009097 16045 raft_consensus.cc:385] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.009274 16045 raft_consensus.cc:740] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 572bfa6ecd224de29c4d5ea51babff36, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.009799 16045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572bfa6ecd224de29c4d5ea51babff36" member_type: VOTER }
I20260709 13:17:08.010221 16045 raft_consensus.cc:399] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:08.010427 16045 raft_consensus.cc:493] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:08.010643 16045 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.014787 16045 raft_consensus.cc:515] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572bfa6ecd224de29c4d5ea51babff36" member_type: VOTER }
I20260709 13:17:08.015326 16045 leader_election.cc:304] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 572bfa6ecd224de29c4d5ea51babff36; no voters: 
I20260709 13:17:08.016513 16045 leader_election.cc:290] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:08.016880 16048 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:08.018322 16048 raft_consensus.cc:697] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 1 LEADER]: Becoming Leader. State: Replica: 572bfa6ecd224de29c4d5ea51babff36, State: Running, Role: LEADER
I20260709 13:17:08.019145 16048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572bfa6ecd224de29c4d5ea51babff36" member_type: VOTER }
I20260709 13:17:08.019672 16045 sys_catalog.cc:565] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:08.023840 16049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "572bfa6ecd224de29c4d5ea51babff36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572bfa6ecd224de29c4d5ea51babff36" member_type: VOTER } }
I20260709 13:17:08.024506 16049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:08.024338 16050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 572bfa6ecd224de29c4d5ea51babff36. Latest consensus state: current_term: 1 leader_uuid: "572bfa6ecd224de29c4d5ea51babff36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "572bfa6ecd224de29c4d5ea51babff36" member_type: VOTER } }
I20260709 13:17:08.024857 16050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:08.026973 16054 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:08.033444 16054 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:08.034500 15476 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:08.041703 16054 catalog_manager.cc:1383] Generated new cluster ID: 95f863b94b0a4cee936df307eae7efcf
I20260709 13:17:08.041987 16054 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:08.058334 16054 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:08.059566 16054 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:08.073982 16054 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36: Generated new TSK 0
I20260709 13:17:08.074563 16054 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:08.101179 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:08.107079 16066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:08.108063 16067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:08.109992 15476 server_base.cc:1061] running on GCE node
W20260709 13:17:08.110347 16069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:08.111344 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:08.111537 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:08.111665 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603028111647 us; error 0 us; skew 500 ppm
I20260709 13:17:08.112154 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:08.114380 15476 webserver.cc:533] Webserver started at http://127.15.29.1:35795/ using document root <none> and password file <none>
I20260709 13:17:08.114820 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:08.115067 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:08.115283 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:08.116264 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-0-root/instance:
uuid: "691fdf9ebed94402a4d88c99ea52d10e"
format_stamp: "Formatted at 2026-07-09 13:17:08 on dist-test-slave-hz8m"
I20260709 13:17:08.120577 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:08.123910 16074 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.124743 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:17:08.125109 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-0-root
uuid: "691fdf9ebed94402a4d88c99ea52d10e"
format_stamp: "Formatted at 2026-07-09 13:17:08 on dist-test-slave-hz8m"
I20260709 13:17:08.125375 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:08.137144 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:08.138365 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:08.139927 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:08.142100 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:08.142318 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.142576 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:08.142758 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.182257 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.1:40529
I20260709 13:17:08.182356 16136 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.1:40529 every 8 connection(s)
I20260709 13:17:08.187678 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:08.195580 16142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:08.196424 16141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:08.198386 16144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:08.198743 15476 server_base.cc:1061] running on GCE node
I20260709 13:17:08.200426 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:08.200719 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:08.201037 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603028200914 us; error 0 us; skew 500 ppm
I20260709 13:17:08.201851 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:08.203817 16137 heartbeater.cc:344] Connected to a master server at 127.15.29.62:33389
I20260709 13:17:08.204150 16137 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:08.204944 16137 heartbeater.cc:507] Master 127.15.29.62:33389 requested a full tablet report, sending...
I20260709 13:17:08.205070 15476 webserver.cc:533] Webserver started at http://127.15.29.2:41715/ using document root <none> and password file <none>
I20260709 13:17:08.205658 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:08.205889 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:08.206213 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:08.207315 16010 ts_manager.cc:194] Registered new tserver with Master: 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529)
I20260709 13:17:08.207706 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-1-root/instance:
uuid: "4d77c04a96e4404c88186359f9ec6bdd"
format_stamp: "Formatted at 2026-07-09 13:17:08 on dist-test-slave-hz8m"
I20260709 13:17:08.209704 16010 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59802
I20260709 13:17:08.213167 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 13:17:08.216324 16149 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.217060 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:17:08.217315 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-1-root
uuid: "4d77c04a96e4404c88186359f9ec6bdd"
format_stamp: "Formatted at 2026-07-09 13:17:08 on dist-test-slave-hz8m"
I20260709 13:17:08.217552 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:08.228112 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:08.229332 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:08.231047 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:08.233245 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:08.233469 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.233732 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:08.233919 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.272933 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.2:38899
I20260709 13:17:08.273038 16211 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.2:38899 every 8 connection(s)
I20260709 13:17:08.277797 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:08.285220 16215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:08.287388 16216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:08.288493 16218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:08.289332 15476 server_base.cc:1061] running on GCE node
I20260709 13:17:08.290426 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:08.290681 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:08.290894 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603028290851 us; error 0 us; skew 500 ppm
I20260709 13:17:08.292495 16212 heartbeater.cc:344] Connected to a master server at 127.15.29.62:33389
I20260709 13:17:08.292677 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:08.292857 16212 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:08.293902 16212 heartbeater.cc:507] Master 127.15.29.62:33389 requested a full tablet report, sending...
I20260709 13:17:08.295486 15476 webserver.cc:533] Webserver started at http://127.15.29.3:37055/ using document root <none> and password file <none>
I20260709 13:17:08.296103 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:08.296335 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:08.296372 16010 ts_manager.cc:194] Registered new tserver with Master: 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:08.296736 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:08.298265 16010 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59814
I20260709 13:17:08.298336 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-2-root/instance:
uuid: "3aaefafdc9a44a6691875b012f3defe1"
format_stamp: "Formatted at 2026-07-09 13:17:08 on dist-test-slave-hz8m"
I20260709 13:17:08.303220 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:17:08.306541 16223 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.307404 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:17:08.307672 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-2-root
uuid: "3aaefafdc9a44a6691875b012f3defe1"
format_stamp: "Formatted at 2026-07-09 13:17:08 on dist-test-slave-hz8m"
I20260709 13:17:08.307917 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783603014893117-15476-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:08.320775 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:08.322190 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:08.323750 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:08.326141 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:08.326326 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.326634 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:08.326794 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:08.367475 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.3:38939
I20260709 13:17:08.367584 16285 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.3:38939 every 8 connection(s)
I20260709 13:17:08.381150 16286 heartbeater.cc:344] Connected to a master server at 127.15.29.62:33389
I20260709 13:17:08.381510 16286 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:08.382236 16286 heartbeater.cc:507] Master 127.15.29.62:33389 requested a full tablet report, sending...
I20260709 13:17:08.384171 16010 ts_manager.cc:194] Registered new tserver with Master: 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:08.385097 15476 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014425509s
I20260709 13:17:08.385530 16010 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59830
I20260709 13:17:08.410627 16010 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59842:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:08.413610 16010 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:08.494102 16102 tablet_service.cc:1511] Processing CreateTablet for tablet 70ddb69735cc469db6aae77c64705301 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:08.495635 16102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ddb69735cc469db6aae77c64705301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.498660 16101 tablet_service.cc:1511] Processing CreateTablet for tablet f3f2358cfe974d0a8ef79e0473ccb918 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:08.500144 16101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f2358cfe974d0a8ef79e0473ccb918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.501490 16249 tablet_service.cc:1511] Processing CreateTablet for tablet 34c39404ff264334a86dbfba157d1fd9 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:08.501843 16098 tablet_service.cc:1511] Processing CreateTablet for tablet 59b3a442dbdc4742a3684d96cb8d7cbd (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:08.502910 16249 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c39404ff264334a86dbfba157d1fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.503262 16098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b3a442dbdc4742a3684d96cb8d7cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.504122 16100 tablet_service.cc:1511] Processing CreateTablet for tablet 34c39404ff264334a86dbfba157d1fd9 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:08.504383 16177 tablet_service.cc:1511] Processing CreateTablet for tablet 70ddb69735cc469db6aae77c64705301 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:08.505788 16177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ddb69735cc469db6aae77c64705301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.498660 16251 tablet_service.cc:1511] Processing CreateTablet for tablet 70ddb69735cc469db6aae77c64705301 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:08.506960 16251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ddb69735cc469db6aae77c64705301. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.506834 16099 tablet_service.cc:1511] Processing CreateTablet for tablet d6371667f79a40f688f1abc3a81506b6 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:08.508093 16176 tablet_service.cc:1511] Processing CreateTablet for tablet f3f2358cfe974d0a8ef79e0473ccb918 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:08.509392 16176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f2358cfe974d0a8ef79e0473ccb918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.509862 16248 tablet_service.cc:1511] Processing CreateTablet for tablet d6371667f79a40f688f1abc3a81506b6 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:08.511270 16248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6371667f79a40f688f1abc3a81506b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.511976 16175 tablet_service.cc:1511] Processing CreateTablet for tablet 34c39404ff264334a86dbfba157d1fd9 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:08.513314 16175 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c39404ff264334a86dbfba157d1fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.499733 16250 tablet_service.cc:1511] Processing CreateTablet for tablet f3f2358cfe974d0a8ef79e0473ccb918 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:08.515762 16250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3f2358cfe974d0a8ef79e0473ccb918. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.516103 16174 tablet_service.cc:1511] Processing CreateTablet for tablet d6371667f79a40f688f1abc3a81506b6 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:08.517453 16174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6371667f79a40f688f1abc3a81506b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.505586 16100 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34c39404ff264334a86dbfba157d1fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.519726 16099 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6371667f79a40f688f1abc3a81506b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.524806 16173 tablet_service.cc:1511] Processing CreateTablet for tablet 59b3a442dbdc4742a3684d96cb8d7cbd (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:08.526067 16247 tablet_service.cc:1511] Processing CreateTablet for tablet 59b3a442dbdc4742a3684d96cb8d7cbd (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:08.526410 16173 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b3a442dbdc4742a3684d96cb8d7cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.527604 16247 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b3a442dbdc4742a3684d96cb8d7cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.531457 16097 tablet_service.cc:1511] Processing CreateTablet for tablet d2b63f8158d341beb37aa11394c9b562 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:08.532989 16097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b63f8158d341beb37aa11394c9b562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.533907 16172 tablet_service.cc:1511] Processing CreateTablet for tablet d2b63f8158d341beb37aa11394c9b562 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:08.535930 16172 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b63f8158d341beb37aa11394c9b562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.535990 16096 tablet_service.cc:1511] Processing CreateTablet for tablet fc5cdb12fb3a4c58ad6399da4179ac36 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:08.537382 16096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc5cdb12fb3a4c58ad6399da4179ac36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.538446 16171 tablet_service.cc:1511] Processing CreateTablet for tablet fc5cdb12fb3a4c58ad6399da4179ac36 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:08.541776 16245 tablet_service.cc:1511] Processing CreateTablet for tablet fc5cdb12fb3a4c58ad6399da4179ac36 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:08.542407 16095 tablet_service.cc:1511] Processing CreateTablet for tablet a244ab737d95446a8f2563ced8410f01 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:08.543316 16245 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc5cdb12fb3a4c58ad6399da4179ac36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.543735 16095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a244ab737d95446a8f2563ced8410f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.545912 16170 tablet_service.cc:1511] Processing CreateTablet for tablet a244ab737d95446a8f2563ced8410f01 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:08.547721 16246 tablet_service.cc:1511] Processing CreateTablet for tablet d2b63f8158d341beb37aa11394c9b562 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:08.548705 16244 tablet_service.cc:1511] Processing CreateTablet for tablet a244ab737d95446a8f2563ced8410f01 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:08.550096 16244 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a244ab737d95446a8f2563ced8410f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.550779 16170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a244ab737d95446a8f2563ced8410f01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.552341 16246 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2b63f8158d341beb37aa11394c9b562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.554243 16094 tablet_service.cc:1511] Processing CreateTablet for tablet 725759fe33b34004ab1573af41da8c94 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:08.555234 16171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc5cdb12fb3a4c58ad6399da4179ac36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.555569 16094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725759fe33b34004ab1573af41da8c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.558667 16169 tablet_service.cc:1511] Processing CreateTablet for tablet 725759fe33b34004ab1573af41da8c94 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:08.560461 16169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725759fe33b34004ab1573af41da8c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.564658 16093 tablet_service.cc:1511] Processing CreateTablet for tablet 36a2dd202cb74d5890cbb655827d79a4 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:08.566103 16093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36a2dd202cb74d5890cbb655827d79a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.568753 16243 tablet_service.cc:1511] Processing CreateTablet for tablet 725759fe33b34004ab1573af41da8c94 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:08.570171 16243 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 725759fe33b34004ab1573af41da8c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.599324 16168 tablet_service.cc:1511] Processing CreateTablet for tablet 36a2dd202cb74d5890cbb655827d79a4 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:08.600730 16168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36a2dd202cb74d5890cbb655827d79a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.600770 16242 tablet_service.cc:1511] Processing CreateTablet for tablet 36a2dd202cb74d5890cbb655827d79a4 (DEFAULT_TABLE table=test-workload [id=ca6792523ca642ffb71dc7038b1c0d79]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:08.604395 16242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36a2dd202cb74d5890cbb655827d79a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:08.648262 16303 tablet_bootstrap.cc:492] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:08.649184 16302 tablet_bootstrap.cc:492] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:08.685761 16303 tablet_bootstrap.cc:654] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.695384 16302 tablet_bootstrap.cc:654] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.705613 16301 tablet_bootstrap.cc:492] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:08.753043 16301 tablet_bootstrap.cc:654] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.757117 16303 tablet_bootstrap.cc:492] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:08.757615 16303 ts_tablet_manager.cc:1403] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.110s	user 0.020s	sys 0.042s
I20260709 13:17:08.758306 16302 tablet_bootstrap.cc:492] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:08.758948 16302 ts_tablet_manager.cc:1403] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.110s	user 0.015s	sys 0.007s
I20260709 13:17:08.760502 16303 raft_consensus.cc:359] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.761101 16303 raft_consensus.cc:385] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.761355 16303 raft_consensus.cc:740] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.761984 16303 consensus_queue.cc:260] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.762732 16302 raft_consensus.cc:359] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.763731 16302 raft_consensus.cc:385] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.763995 16286 heartbeater.cc:499] Master 127.15.29.62:33389 was elected leader, sending a full tablet report...
I20260709 13:17:08.764179 16302 raft_consensus.cc:740] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.765205 16302 consensus_queue.cc:260] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.767526 16303 ts_tablet_manager.cc:1434] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:17:08.768559 16303 tablet_bootstrap.cc:492] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:08.776413 16303 tablet_bootstrap.cc:654] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.778491 16301 tablet_bootstrap.cc:492] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:08.779196 16301 ts_tablet_manager.cc:1403] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.074s	user 0.008s	sys 0.025s
I20260709 13:17:08.780352 16212 heartbeater.cc:499] Master 127.15.29.62:33389 was elected leader, sending a full tablet report...
I20260709 13:17:08.780827 16302 ts_tablet_manager.cc:1434] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.021s	user 0.017s	sys 0.004s
I20260709 13:17:08.781746 16302 tablet_bootstrap.cc:492] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:08.782217 16301 raft_consensus.cc:359] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.782948 16301 raft_consensus.cc:385] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.783232 16301 raft_consensus.cc:740] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.784108 16309 raft_consensus.cc:493] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.788175 16302 tablet_bootstrap.cc:654] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.788255 16303 tablet_bootstrap.cc:492] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:08.788777 16303 ts_tablet_manager.cc:1403] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.002s
I20260709 13:17:08.790958 16303 raft_consensus.cc:359] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.791684 16303 raft_consensus.cc:385] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.791931 16303 raft_consensus.cc:740] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.792677 16303 consensus_queue.cc:260] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.794623 16303 ts_tablet_manager.cc:1434] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:08.795544 16303 tablet_bootstrap.cc:492] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:08.801914 16303 tablet_bootstrap.cc:654] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.784566 16309 raft_consensus.cc:515] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.813092 16301 consensus_queue.cc:260] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.817061 16309 leader_election.cc:290] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:08.822917 16137 heartbeater.cc:499] Master 127.15.29.62:33389 was elected leader, sending a full tablet report...
I20260709 13:17:08.827046 16301 ts_tablet_manager.cc:1434] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.004s
I20260709 13:17:08.832721 16301 tablet_bootstrap.cc:492] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:08.845577 16307 raft_consensus.cc:493] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.846117 16307 raft_consensus.cc:515] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.848930 16307 leader_election.cc:290] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:08.861634 16302 tablet_bootstrap.cc:492] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:08.862174 16302 ts_tablet_manager.cc:1403] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.081s	user 0.029s	sys 0.040s
I20260709 13:17:08.865253 16302 raft_consensus.cc:359] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.866074 16302 raft_consensus.cc:385] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.866406 16302 raft_consensus.cc:740] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.867327 16302 consensus_queue.cc:260] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.868191 16303 tablet_bootstrap.cc:492] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:08.868697 16303 ts_tablet_manager.cc:1403] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.073s	user 0.024s	sys 0.036s
I20260709 13:17:08.871732 16302 ts_tablet_manager.cc:1434] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:17:08.872907 16302 tablet_bootstrap.cc:492] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:08.877247 16307 raft_consensus.cc:493] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.877182 16303 raft_consensus.cc:359] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.877996 16303 raft_consensus.cc:385] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.877812 16307 raft_consensus.cc:515] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.878315 16303 raft_consensus.cc:740] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.879968 16307 leader_election.cc:290] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:08.884073 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:08.885200 16301 tablet_bootstrap.cc:654] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:08.885697 16153 leader_election.cc:343] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:08.886237 16302 tablet_bootstrap.cc:654] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.882057 16303 consensus_queue.cc:260] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.892313 16303 ts_tablet_manager.cc:1434] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 13:17:08.894831 16303 tablet_bootstrap.cc:492] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:08.895380 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:08.895910 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:08.896126 16261 raft_consensus.cc:2468] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:08.897437 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:08.897712 16153 leader_election.cc:304] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 691fdf9ebed94402a4d88c99ea52d10e
W20260709 13:17:08.898737 16227 leader_election.cc:343] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:08.899807 16309 raft_consensus.cc:2804] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:08.900194 16309 raft_consensus.cc:493] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.900537 16309 raft_consensus.cc:3060] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.901474 16303 tablet_bootstrap.cc:654] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.904837 16302 tablet_bootstrap.cc:492] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:08.905547 16302 ts_tablet_manager.cc:1403] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.006s
I20260709 13:17:08.906558 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:08.907671 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
W20260709 13:17:08.907850 16227 leader_election.cc:343] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:08.909075 16225 leader_election.cc:343] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:08.909178 16309 raft_consensus.cc:515] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.909524 16225 leader_election.cc:304] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1; no voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:08.909997 16302 raft_consensus.cc:359] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.911039 16302 raft_consensus.cc:385] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.911518 16302 raft_consensus.cc:740] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.911240 16187 raft_consensus.cc:2393] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3aaefafdc9a44a6691875b012f3defe1 in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:08.911986 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
W20260709 13:17:08.913244 16153 leader_election.cc:343] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:08.912705 16302 consensus_queue.cc:260] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.910629 16307 raft_consensus.cc:2749] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:08.916525 16225 leader_election.cc:304] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1; no voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:08.917688 16307 raft_consensus.cc:3060] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.917582 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:08.918916 16309 leader_election.cc:290] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:08.921419 16302 ts_tablet_manager.cc:1434] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:17:08.922912 16302 tablet_bootstrap.cc:492] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:08.925918 16307 raft_consensus.cc:2749] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:08.932914 16302 tablet_bootstrap.cc:654] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.934695 16303 tablet_bootstrap.cc:492] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:08.935011 16261 raft_consensus.cc:2468] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 1.
I20260709 13:17:08.935266 16303 ts_tablet_manager.cc:1403] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.015s
I20260709 13:17:08.937208 16153 leader_election.cc:304] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:08.938067 16309 raft_consensus.cc:2804] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:08.938571 16309 raft_consensus.cc:697] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:08.939380 16303 raft_consensus.cc:359] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.939445 16309 consensus_queue.cc:237] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.940202 16303 raft_consensus.cc:385] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.940519 16303 raft_consensus.cc:740] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.941557 16301 tablet_bootstrap.cc:492] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:08.941334 16303 consensus_queue.cc:260] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.942312 16301 ts_tablet_manager.cc:1403] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.110s	user 0.023s	sys 0.010s
I20260709 13:17:08.944168 16303 ts_tablet_manager.cc:1434] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:17:08.945366 16303 tablet_bootstrap.cc:492] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:08.946622 16301 raft_consensus.cc:359] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.947546 16301 raft_consensus.cc:385] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.947867 16301 raft_consensus.cc:740] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.947928 16314 raft_consensus.cc:493] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.948524 16314 raft_consensus.cc:515] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.948933 16301 consensus_queue.cc:260] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.953506 16303 tablet_bootstrap.cc:654] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.954373 16302 tablet_bootstrap.cc:492] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:08.954897 16302 ts_tablet_manager.cc:1403] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.009s
I20260709 13:17:08.958189 16302 raft_consensus.cc:359] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.958987 16302 raft_consensus.cc:385] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.959779 16302 raft_consensus.cc:740] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.959065 16314 leader_election.cc:290] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:08.960752 16302 consensus_queue.cc:260] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.963510 16301 ts_tablet_manager.cc:1434] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.005s
I20260709 13:17:08.964476 16301 tablet_bootstrap.cc:492] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:08.965548 16302 ts_tablet_manager.cc:1434] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:17:08.966596 16302 tablet_bootstrap.cc:492] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:08.970335 16307 raft_consensus.cc:493] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:08.970825 16307 raft_consensus.cc:515] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.972975 16307 leader_election.cc:290] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:08.972227 16010 catalog_manager.cc:5697] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 1 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:08.973235 16302 tablet_bootstrap.cc:654] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.977459 16301 tablet_bootstrap.cc:654] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:08.978677 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:08.981153 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:08.981957 16186 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
W20260709 13:17:08.983037 16227 leader_election.cc:343] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:08.983994 16076 leader_election.cc:304] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:08.984500 16303 tablet_bootstrap.cc:492] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:08.985008 16303 ts_tablet_manager.cc:1403] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.040s	user 0.025s	sys 0.008s
I20260709 13:17:08.985037 16324 raft_consensus.cc:2804] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:08.985517 16324 raft_consensus.cc:493] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:08.985713 16302 tablet_bootstrap.cc:492] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:08.985915 16324 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:08.986450 16302 ts_tablet_manager.cc:1403] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.008s
I20260709 13:17:08.988343 16303 raft_consensus.cc:359] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.989130 16303 raft_consensus.cc:385] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:08.989483 16303 raft_consensus.cc:740] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.990428 16303 consensus_queue.cc:260] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.990553 16302 raft_consensus.cc:359] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.991616 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:08.991634 16302 raft_consensus.cc:385] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:08.993254 16225 leader_election.cc:343] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:08.993772 16225 leader_election.cc:304] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1; no voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:08.994077 16303 ts_tablet_manager.cc:1434] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:17:08.994906 16307 raft_consensus.cc:2749] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:08.995102 16301 tablet_bootstrap.cc:492] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:08.995165 16303 tablet_bootstrap.cc:492] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:08.995694 16302 raft_consensus.cc:740] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:08.996661 16302 consensus_queue.cc:260] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:08.999374 16302 ts_tablet_manager.cc:1434] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 13:17:09.000408 16302 tablet_bootstrap.cc:492] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:08.995702 16301 ts_tablet_manager.cc:1403] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.010s
I20260709 13:17:09.007723 16302 tablet_bootstrap.cc:654] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.009227 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.010274 16301 raft_consensus.cc:359] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.010349 16324 raft_consensus.cc:515] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
W20260709 13:17:09.010926 16078 leader_election.cc:343] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.011144 16301 raft_consensus.cc:385] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.013414 16303 tablet_bootstrap.cc:654] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.011641 16301 raft_consensus.cc:740] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.013583 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.014389 16187 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.015637 16301 consensus_queue.cc:260] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.018388 16301 ts_tablet_manager.cc:1434] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:09.020084 16302 tablet_bootstrap.cc:492] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:09.020656 16302 ts_tablet_manager.cc:1403] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.013s
I20260709 13:17:09.020972 16301 tablet_bootstrap.cc:492] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.024209 16302 raft_consensus.cc:359] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.025074 16302 raft_consensus.cc:385] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.025386 16302 raft_consensus.cc:740] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.026394 16302 consensus_queue.cc:260] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.028789 16302 ts_tablet_manager.cc:1434] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 13:17:09.029762 16302 tablet_bootstrap.cc:492] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:09.035184 16187 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.039191 16301 tablet_bootstrap.cc:654] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.039271 16076 leader_election.cc:304] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.039608 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.035908 16314 raft_consensus.cc:493] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.040796 16314 raft_consensus.cc:515] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.042244 16302 tablet_bootstrap.cc:654] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.037328 16324 leader_election.cc:290] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.043375 16324 raft_consensus.cc:2804] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.043825 16324 raft_consensus.cc:697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.044615 16324 consensus_queue.cc:237] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
W20260709 13:17:09.048627 16058 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:09.049054 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2358cfe974d0a8ef79e0473ccb918" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
W20260709 13:17:09.050941 16076 leader_election.cc:343] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.051481 16301 tablet_bootstrap.cc:492] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.051977 16301 ts_tablet_manager.cc:1403] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.031s	user 0.006s	sys 0.012s
W20260709 13:17:09.054720 16078 leader_election.cc:343] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.055205 16301 raft_consensus.cc:359] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.055953 16301 raft_consensus.cc:385] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.056018 16314 leader_election.cc:290] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.056352 16301 raft_consensus.cc:740] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.057788 16301 consensus_queue.cc:260] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.059341 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2358cfe974d0a8ef79e0473ccb918" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.060040 16261 raft_consensus.cc:2468] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.061149 16302 tablet_bootstrap.cc:492] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:09.061219 16303 tablet_bootstrap.cc:492] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:09.061368 16078 leader_election.cc:304] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.061766 16302 ts_tablet_manager.cc:1403] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.007s
I20260709 13:17:09.061848 16303 ts_tablet_manager.cc:1403] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.067s	user 0.006s	sys 0.013s
I20260709 13:17:09.062760 16334 raft_consensus.cc:2804] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.063170 16334 raft_consensus.cc:493] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.063494 16334 raft_consensus.cc:3060] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.065158 16302 raft_consensus.cc:359] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.066087 16302 raft_consensus.cc:385] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.066404 16302 raft_consensus.cc:740] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.067166 16302 consensus_queue.cc:260] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.069025 16301 ts_tablet_manager.cc:1434] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:17:09.069410 16302 ts_tablet_manager.cc:1434] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:09.070058 16301 tablet_bootstrap.cc:492] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.070354 16302 tablet_bootstrap.cc:492] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:09.070833 16334 raft_consensus.cc:515] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.067394 16303 raft_consensus.cc:359] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.072619 16303 raft_consensus.cc:385] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.072996 16303 raft_consensus.cc:740] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.073765 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2358cfe974d0a8ef79e0473ccb918" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.073943 16303 consensus_queue.cc:260] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
W20260709 13:17:09.075428 16076 leader_election.cc:343] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.077804 16302 tablet_bootstrap.cc:654] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.077759 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2358cfe974d0a8ef79e0473ccb918" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.078470 16261 raft_consensus.cc:3060] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.085762 16261 raft_consensus.cc:2468] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.087082 16303 ts_tablet_manager.cc:1434] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.002s
I20260709 13:17:09.087170 16078 leader_election.cc:304] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.088454 16301 tablet_bootstrap.cc:654] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.089207 16314 raft_consensus.cc:2804] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.087832 16334 leader_election.cc:290] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.098172 16303 tablet_bootstrap.cc:492] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:09.111976 16334 raft_consensus.cc:493] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.112561 16334 raft_consensus.cc:515] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.115072 16309 raft_consensus.cc:493] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.117393 16309 raft_consensus.cc:515] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.119691 16309 leader_election.cc:290] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.120204 16340 raft_consensus.cc:493] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.120841 16340 raft_consensus.cc:515] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.124846 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:09.125425 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.125627 16112 raft_consensus.cc:2468] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.126242 16261 raft_consensus.cc:2468] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.127781 16153 leader_election.cc:304] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:09.133332 16309 raft_consensus.cc:2804] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.133843 16309 raft_consensus.cc:493] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.134434 16309 raft_consensus.cc:3060] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.124861 16008 catalog_manager.cc:5697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.140313 16334 leader_election.cc:290] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.130548 16314 raft_consensus.cc:697] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.144644 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.145543 16261 raft_consensus.cc:2468] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.147404 16078 leader_election.cc:304] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.150827 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.151811 16314 consensus_queue.cc:237] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.154011 16334 raft_consensus.cc:2804] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.154502 16334 raft_consensus.cc:493] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.154951 16334 raft_consensus.cc:3060] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.157012 16303 tablet_bootstrap.cc:654] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.171365 16303 tablet_bootstrap.cc:492] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:09.171522 16301 tablet_bootstrap.cc:492] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.171898 16303 ts_tablet_manager.cc:1403] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.003s
I20260709 13:17:09.172329 16301 ts_tablet_manager.cc:1403] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.102s	user 0.041s	sys 0.041s
I20260709 13:17:09.172915 16302 tablet_bootstrap.cc:492] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:09.173365 16302 ts_tablet_manager.cc:1403] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.103s	user 0.027s	sys 0.048s
I20260709 13:17:09.174033 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.174257 16309 raft_consensus.cc:515] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.175426 16187 raft_consensus.cc:2393] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 691fdf9ebed94402a4d88c99ea52d10e in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:09.174957 16303 raft_consensus.cc:359] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.175823 16303 raft_consensus.cc:385] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.176163 16303 raft_consensus.cc:740] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.175984 16302 raft_consensus.cc:359] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.176652 16302 raft_consensus.cc:385] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:09.175845 16076 leader_election.cc:343] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.176952 16302 raft_consensus.cc:740] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.177843 16302 consensus_queue.cc:260] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.178283 16303 consensus_queue.cc:260] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.180171 16302 ts_tablet_manager.cc:1434] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:17:09.181165 16302 tablet_bootstrap.cc:492] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:09.183791 16334 raft_consensus.cc:515] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.184510 16308 raft_consensus.cc:493] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.184983 16308 raft_consensus.cc:515] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.187098 16302 tablet_bootstrap.cc:654] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.187259 16308 leader_election.cc:290] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.187736 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.188367 16261 raft_consensus.cc:2468] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.189263 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.190024 16260 raft_consensus.cc:2468] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.190304 16078 leader_election.cc:304] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.191330 16341 raft_consensus.cc:2804] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.191720 16341 raft_consensus.cc:493] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.192057 16341 raft_consensus.cc:3060] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.193198 16153 leader_election.cc:304] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:09.194145 16308 raft_consensus.cc:2804] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.194365 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:09.195574 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.178560 16301 raft_consensus.cc:359] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.196369 16187 raft_consensus.cc:2393] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 691fdf9ebed94402a4d88c99ea52d10e in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:09.194532 16308 raft_consensus.cc:493] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.197269 16308 raft_consensus.cc:3060] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.200348 16309 leader_election.cc:290] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.198429 16340 leader_election.cc:290] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.201797 16008 catalog_manager.cc:5697] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.202961 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.203591 16260 raft_consensus.cc:3060] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.204421 16301 raft_consensus.cc:385] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:09.206226 16153 leader_election.cc:343] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.208487 16301 raft_consensus.cc:740] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.208750 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:09.209532 16112 raft_consensus.cc:2393] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d77c04a96e4404c88186359f9ec6bdd in current term 1: Already voted for candidate 691fdf9ebed94402a4d88c99ea52d10e in this term.
I20260709 13:17:09.209685 16301 consensus_queue.cc:260] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.210985 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.212200 16301 ts_tablet_manager.cc:1434] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.039s	user 0.005s	sys 0.002s
I20260709 13:17:09.213302 16301 tablet_bootstrap.cc:492] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.213619 16153 leader_election.cc:304] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd; no voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.215152 16309 raft_consensus.cc:493] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.215788 16309 raft_consensus.cc:515] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.216817 16303 ts_tablet_manager.cc:1434] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:17:09.207832 16334 leader_election.cc:290] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.206251 16308 raft_consensus.cc:515] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.220661 16309 leader_election.cc:290] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.224006 16301 tablet_bootstrap.cc:654] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.226311 16309 raft_consensus.cc:2749] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.228119 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:09.228282 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.228937 16261 raft_consensus.cc:2468] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.230672 16303 tablet_bootstrap.cc:492] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:09.231752 16308 leader_election.cc:290] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.232585 16153 leader_election.cc:304] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:09.233402 16308 raft_consensus.cc:2804] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.233641 16341 raft_consensus.cc:515] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.234906 16112 raft_consensus.cc:2393] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d77c04a96e4404c88186359f9ec6bdd in current term 1: Already voted for candidate 691fdf9ebed94402a4d88c99ea52d10e in this term.
I20260709 13:17:09.234642 16111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:09.233846 16308 raft_consensus.cc:493] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:09.238169 16153 leader_election.cc:343] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.238582 16308 raft_consensus.cc:3060] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.237203 16341 leader_election.cc:290] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.237310 16302 tablet_bootstrap.cc:492] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:09.239558 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.240453 16302 ts_tablet_manager.cc:1403] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.023s
I20260709 13:17:09.237588 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.240398 16259 raft_consensus.cc:3060] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.240995 16261 raft_consensus.cc:3060] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.237592 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.243331 16302 raft_consensus.cc:359] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.244062 16302 raft_consensus.cc:385] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.244350 16302 raft_consensus.cc:740] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.245062 16302 consensus_queue.cc:260] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.247759 16309 raft_consensus.cc:493] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.248351 16309 raft_consensus.cc:515] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.249619 16302 ts_tablet_manager.cc:1434] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:17:09.250590 16302 tablet_bootstrap.cc:492] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: Bootstrap starting.
I20260709 13:17:09.251940 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ddb69735cc469db6aae77c64705301" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
W20260709 13:17:09.254279 16153 leader_election.cc:343] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.255208 16261 raft_consensus.cc:2468] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.241153 16260 raft_consensus.cc:2468] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.256997 16302 tablet_bootstrap.cc:654] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.260201 16078 leader_election.cc:304] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.261003 16341 raft_consensus.cc:2804] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.261267 16078 leader_election.cc:304] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.261570 16341 raft_consensus.cc:697] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.261876 16308 raft_consensus.cc:515] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.262542 16334 raft_consensus.cc:2804] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.259420 16309 leader_election.cc:290] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.262461 16341 consensus_queue.cc:237] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.263242 16334 raft_consensus.cc:697] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.264038 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ddb69735cc469db6aae77c64705301" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.264294 16334 consensus_queue.cc:237] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
W20260709 13:17:09.265638 16153 leader_election.cc:343] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.266181 16153 leader_election.cc:304] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd; no voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.267150 16259 raft_consensus.cc:2468] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 1.
I20260709 13:17:09.271981 16301 tablet_bootstrap.cc:492] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.272984 16301 ts_tablet_manager.cc:1403] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.013s
I20260709 13:17:09.273882 16153 leader_election.cc:304] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.274745 16309 raft_consensus.cc:2749] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.276981 16346 raft_consensus.cc:2804] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.277586 16346 raft_consensus.cc:697] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:09.277756 16301 raft_consensus.cc:359] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.278496 16346 consensus_queue.cc:237] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.279448 16187 raft_consensus.cc:2393] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 691fdf9ebed94402a4d88c99ea52d10e in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:09.278483 16301 raft_consensus.cc:385] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.281059 16301 raft_consensus.cc:740] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.281665 16303 tablet_bootstrap.cc:654] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.281934 16301 consensus_queue.cc:260] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.285650 16301 ts_tablet_manager.cc:1434] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.005s
I20260709 13:17:09.286724 16301 tablet_bootstrap.cc:492] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.290568 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:09.294376 16301 tablet_bootstrap.cc:654] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.294772 16308 leader_election.cc:290] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.299408 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.300616 16260 raft_consensus.cc:2393] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d77c04a96e4404c88186359f9ec6bdd in current term 1: Already voted for candidate 691fdf9ebed94402a4d88c99ea52d10e in this term.
I20260709 13:17:09.302202 16153 leader_election.cc:304] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd; no voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.303820 16308 raft_consensus.cc:2749] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.312177 16301 tablet_bootstrap.cc:492] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.312757 16301 ts_tablet_manager.cc:1403] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.026s	user 0.006s	sys 0.005s
I20260709 13:17:09.313052 16308 raft_consensus.cc:493] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.313647 16302 tablet_bootstrap.cc:492] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: No bootstrap required, opened a new log
I20260709 13:17:09.314097 16302 ts_tablet_manager.cc:1403] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.018s
I20260709 13:17:09.313649 16308 raft_consensus.cc:515] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.316260 16301 raft_consensus.cc:359] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.317561 16301 raft_consensus.cc:385] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.317910 16301 raft_consensus.cc:740] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.315925 16008 catalog_manager.cc:5697] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.318733 16301 consensus_queue.cc:260] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.319176 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:09.319832 16112 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.321192 16308 leader_election.cc:290] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.321117 16153 leader_election.cc:304] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.321794 16303 tablet_bootstrap.cc:492] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:09.322728 16302 raft_consensus.cc:359] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.323436 16302 raft_consensus.cc:385] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.323688 16302 raft_consensus.cc:740] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.324121 16301 ts_tablet_manager.cc:1434] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:17:09.324608 16308 raft_consensus.cc:2804] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.324970 16308 raft_consensus.cc:493] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.325143 16301 tablet_bootstrap.cc:492] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.325376 16308 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.325268 16007 catalog_manager.cc:5697] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 1 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.329864 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.330835 16260 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.332962 16301 tablet_bootstrap.cc:654] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.332618 16308 raft_consensus.cc:515] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.335311 16302 consensus_queue.cc:260] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.335618 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:09.336424 16112 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.337496 16302 ts_tablet_manager.cc:1434] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 13:17:09.339079 16308 leader_election.cc:290] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.322234 16303 ts_tablet_manager.cc:1403] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.001s
I20260709 13:17:09.344023 16303 raft_consensus.cc:359] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.344715 16303 raft_consensus.cc:385] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.344985 16303 raft_consensus.cc:740] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.346176 16341 raft_consensus.cc:493] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.346654 16341 raft_consensus.cc:515] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.348003 16008 catalog_manager.cc:5697] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.348968 16341 leader_election.cc:290] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.350636 16346 consensus_queue.cc:1048] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.351157 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.354728 16187 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.356014 16076 leader_election.cc:304] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.351877 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.345631 16303 consensus_queue.cc:260] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.353309 16112 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 1.
I20260709 13:17:09.353777 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.360555 16259 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.361371 16341 raft_consensus.cc:2804] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.363572 16341 raft_consensus.cc:493] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.363951 16341 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.364113 16153 leader_election.cc:304] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.366019 16303 ts_tablet_manager.cc:1434] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:17:09.367046 16303 tablet_bootstrap.cc:492] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:09.367273 16260 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.369095 16346 raft_consensus.cc:2804] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.369616 16346 raft_consensus.cc:697] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:09.373615 16309 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.374315 16309 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.376199 16303 tablet_bootstrap.cc:654] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.370927 16346 consensus_queue.cc:237] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.378669 16309 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.378777 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
W20260709 13:17:09.381227 16153 leader_election.cc:343] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.380962 16341 raft_consensus.cc:515] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.383033 16301 tablet_bootstrap.cc:492] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.383003 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.385788 16259 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 0.
I20260709 13:17:09.384831 16260 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 1.
I20260709 13:17:09.387054 16301 ts_tablet_manager.cc:1403] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.062s	user 0.016s	sys 0.034s
I20260709 13:17:09.388358 16153 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.388738 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.389402 16187 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.389791 16308 raft_consensus.cc:2804] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.390280 16301 raft_consensus.cc:359] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.390758 16301 raft_consensus.cc:385] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.391124 16301 raft_consensus.cc:740] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.391742 16301 consensus_queue.cc:260] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.390170 16308 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.393271 16308 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.434393 16301 ts_tablet_manager.cc:1434] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 13:17:09.435514 16301 tablet_bootstrap.cc:492] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.431785 16340 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.436846 16340 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.439538 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.439841 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.440160 16260 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.440959 16340 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.441967 16301 tablet_bootstrap.cc:654] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.443390 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.444427 16259 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.447321 16260 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.447489 16078 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.447721 16187 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.448791 16076 leader_election.cc:304] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.436193 16341 leader_election.cc:290] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.449522 16314 raft_consensus.cc:2804] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.449580 16341 raft_consensus.cc:2804] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.449847 16334 raft_consensus.cc:493] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.450448 16314 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.453089 16334 raft_consensus.cc:515] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.453613 16314 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.453737 16007 catalog_manager.cc:5697] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 1 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.456151 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ddb69735cc469db6aae77c64705301" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.456609 16303 tablet_bootstrap.cc:492] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:09.457211 16303 ts_tablet_manager.cc:1403] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.090s	user 0.013s	sys 0.019s
I20260709 13:17:09.466013 16303 raft_consensus.cc:359] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.467065 16303 raft_consensus.cc:385] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.467435 16303 raft_consensus.cc:740] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.470732 16314 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.468252 16341 raft_consensus.cc:697] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.452178 16308 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.456769 16187 raft_consensus.cc:2468] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 0.
I20260709 13:17:09.473671 16314 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.476472 16303 consensus_queue.cc:260] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.479326 16334 leader_election.cc:290] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.479996 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.480297 16186 raft_consensus.cc:2393] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 691fdf9ebed94402a4d88c99ea52d10e in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:09.480571 16259 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.480850 16258 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ddb69735cc469db6aae77c64705301" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.481779 16341 consensus_queue.cc:237] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.484277 16308 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
W20260709 13:17:09.485334 16078 leader_election.cc:343] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:09.485623 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:09.491559 16076 leader_election.cc:304] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:09.492355 16259 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.493361 16303 ts_tablet_manager.cc:1434] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.004s
I20260709 13:17:09.492946 16334 raft_consensus.cc:2804] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:09.494650 16303 tablet_bootstrap.cc:492] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1: Bootstrap starting.
I20260709 13:17:09.493889 16112 raft_consensus.cc:2393] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d77c04a96e4404c88186359f9ec6bdd in current term 1: Already voted for candidate 691fdf9ebed94402a4d88c99ea52d10e in this term.
W20260709 13:17:09.494830 16078 consensus_peers.cc:597] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e -> Peer 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939): Couldn't send request to peer 3aaefafdc9a44a6691875b012f3defe1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:09.497880 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.498690 16186 raft_consensus.cc:2393] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 691fdf9ebed94402a4d88c99ea52d10e in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:09.499079 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.497898 16078 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.500093 16314 raft_consensus.cc:2804] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.500058 16260 raft_consensus.cc:2393] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4d77c04a96e4404c88186359f9ec6bdd in current term 1: Already voted for candidate 691fdf9ebed94402a4d88c99ea52d10e in this term.
I20260709 13:17:09.494365 16334 raft_consensus.cc:493] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:09.501052 16334 raft_consensus.cc:3060] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.501644 16314 raft_consensus.cc:697] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.503372 16303 tablet_bootstrap.cc:654] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.505064 16314 consensus_queue.cc:237] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
W20260709 13:17:09.494983 16153 consensus_peers.cc:597] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd -> Peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Couldn't send request to peer 691fdf9ebed94402a4d88c99ea52d10e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:09.511327 16359 raft_consensus.cc:493] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.512213 16153 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd; no voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.512991 16303 tablet_bootstrap.cc:492] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1: No bootstrap required, opened a new log
I20260709 13:17:09.513042 16308 raft_consensus.cc:2749] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.513482 16303 ts_tablet_manager.cc:1403] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.001s
I20260709 13:17:09.511780 16359 raft_consensus.cc:515] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.515156 16359 leader_election.cc:290] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.515419 16303 raft_consensus.cc:359] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.516094 16303 raft_consensus.cc:385] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.516330 16303 raft_consensus.cc:740] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.516961 16303 consensus_queue.cc:260] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.519054 16303 ts_tablet_manager.cc:1434] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:17:09.519482 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2358cfe974d0a8ef79e0473ccb918" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:09.521520 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3f2358cfe974d0a8ef79e0473ccb918" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.522303 16260 raft_consensus.cc:2393] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4d77c04a96e4404c88186359f9ec6bdd in current term 1: Already voted for candidate 691fdf9ebed94402a4d88c99ea52d10e in this term.
I20260709 13:17:09.523504 16301 tablet_bootstrap.cc:492] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.521494 16334 raft_consensus.cc:515] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.523680 16153 leader_election.cc:304] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd; no voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.524070 16301 ts_tablet_manager.cc:1403] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.000s
I20260709 13:17:09.524742 16359 raft_consensus.cc:3060] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.526046 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ddb69735cc469db6aae77c64705301" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.526790 16334 leader_election.cc:290] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.527652 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70ddb69735cc469db6aae77c64705301" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.528088 16260 raft_consensus.cc:3060] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.531402 16186 raft_consensus.cc:3060] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.531355 16334 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.531587 16301 raft_consensus.cc:359] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.526386 16008 catalog_manager.cc:5697] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.532327 16301 raft_consensus.cc:385] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.532613 16301 raft_consensus.cc:740] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.538587 16186 raft_consensus.cc:2468] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.533329 16301 consensus_queue.cc:260] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.543902 16076 leader_election.cc:304] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:09.544399 16301 ts_tablet_manager.cc:1434] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 13:17:09.544813 16359 raft_consensus.cc:2749] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.545326 16301 tablet_bootstrap.cc:492] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e: Bootstrap starting.
I20260709 13:17:09.545372 16341 raft_consensus.cc:2804] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:09.546092 16341 raft_consensus.cc:697] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:09.542434 16260 raft_consensus.cc:2468] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 1.
I20260709 13:17:09.547171 16341 consensus_queue.cc:237] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.560338 16009 catalog_manager.cc:5697] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.564970 16301 tablet_bootstrap.cc:654] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:09.571457 16301 tablet_bootstrap.cc:492] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e: No bootstrap required, opened a new log
I20260709 13:17:09.571914 16301 ts_tablet_manager.cc:1403] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 13:17:09.574453 16301 raft_consensus.cc:359] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.575444 16301 raft_consensus.cc:385] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:09.575722 16301 raft_consensus.cc:740] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Initialized, Role: FOLLOWER
I20260709 13:17:09.576313 16301 consensus_queue.cc:260] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.577811 16301 ts_tablet_manager.cc:1434] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:09.577869 16009 catalog_manager.cc:5697] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 0 to 1, leader changed from <none> to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 1 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:09.592317 16340 consensus_queue.cc:1048] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.601939 16314 consensus_queue.cc:1048] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.616716 16314 raft_consensus.cc:493] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.617298 16314 raft_consensus.cc:515] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.619609 16314 leader_election.cc:290] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.620406 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.620631 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.621311 16260 raft_consensus.cc:2393] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 691fdf9ebed94402a4d88c99ea52d10e in current term 1: Already voted for candidate 4d77c04a96e4404c88186359f9ec6bdd in this term.
I20260709 13:17:09.622299 16078 leader_election.cc:304] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 691fdf9ebed94402a4d88c99ea52d10e; no voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.623044 16314 raft_consensus.cc:3060] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.627931 16314 raft_consensus.cc:2749] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.676725 16314 consensus_queue.cc:1048] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.684020 16341 consensus_queue.cc:1048] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.687285 16307 raft_consensus.cc:493] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.687815 16307 raft_consensus.cc:515] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.687968 16340 consensus_queue.cc:1048] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.689972 16307 leader_election.cc:290] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:09.690610 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e" is_pre_election: true
I20260709 13:17:09.691752 16185 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.693223 16225 leader_election.cc:304] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1; no voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.694267 16307 raft_consensus.cc:2749] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.695004 16341 consensus_queue.cc:1048] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.740978 16186 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
mode: GRACEFUL
new_leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:32974
I20260709 13:17:09.741472 16186 raft_consensus.cc:606] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Received request to transfer leadership to TS 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:09.762650 16112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:44650
I20260709 13:17:09.763207 16112 raft_consensus.cc:606] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Received request to transfer leadership to TS 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.768049 16112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "725759fe33b34004ab1573af41da8c94"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:44650
I20260709 13:17:09.768507 16112 raft_consensus.cc:606] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Received request to transfer leadership to TS 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.773257 16112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:44650
I20260709 13:17:09.773729 16112 raft_consensus.cc:606] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Received request to transfer leadership to TS 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:09.852727 16359 consensus_queue.cc:1048] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.863435 16359 consensus_queue.cc:1048] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.913143 16340 raft_consensus.cc:493] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:09.913647 16340 raft_consensus.cc:515] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.915371 16340 leader_election.cc:290] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.916239 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd" is_pre_election: true
I20260709 13:17:09.916363 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaefafdc9a44a6691875b012f3defe1" is_pre_election: true
I20260709 13:17:09.917709 16078 leader_election.cc:304] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 691fdf9ebed94402a4d88c99ea52d10e; no voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:09.918334 16340 raft_consensus.cc:2749] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:09.935752 16260 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:09.940209 16340 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.945987 16340 consensus_queue.cc:1048] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.946230 16359 consensus_queue.cc:1048] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.964210 16359 raft_consensus.cc:993] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: : Instructing follower 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:09.964670 16359 raft_consensus.cc:1081] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Signalling peer 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:09.967357 16111 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:44630
I20260709 13:17:09.967929 16111 raft_consensus.cc:493] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:09.968317 16111 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:09.970690 16340 consensus_queue.cc:1048] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.970767 16308 consensus_queue.cc:1048] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:09.978291 16111 raft_consensus.cc:515] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.980671 16111 leader_election.cc:290] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 2 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:09.983628 16112 raft_consensus.cc:1240] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Rejecting Update request from peer 4d77c04a96e4404c88186359f9ec6bdd for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:09.985145 16340 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.986207 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:09.988960 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:09.989670 16309 consensus_queue.cc:1059] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:09.989674 16186 raft_consensus.cc:3055] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:09.996204 16186 raft_consensus.cc:740] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:09.997433 16186 consensus_queue.cc:260] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:09.999174 16186 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.007495 16186 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 2.
W20260709 13:17:10.008042 16309 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 13:17:10.009454 16076 leader_election.cc:304] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:10.010324 16340 raft_consensus.cc:2804] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:10.010887 16340 raft_consensus.cc:697] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:10.011682 16340 consensus_queue.cc:237] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.019992 16010 catalog_manager.cc:5697] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 1 to 2, leader changed from 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2) to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.025077 16341 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:10.032217 16340 raft_consensus.cc:993] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:10.032706 16334 raft_consensus.cc:1081] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Signalling peer 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:10.034641 16187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:32970
I20260709 13:17:10.035509 16187 raft_consensus.cc:493] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.035913 16187 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.039399 16340 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:10.041401 16187 raft_consensus.cc:515] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.044363 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:10.044966 16112 raft_consensus.cc:3055] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:10.045369 16112 raft_consensus.cc:740] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:10.046188 16112 consensus_queue.cc:260] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.047187 16112 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.054468 16112 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 2.
I20260709 13:17:10.055472 16187 leader_election.cc:290] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 2 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:10.056078 16153 leader_election.cc:304] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:10.056951 16261 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:10.058657 16309 raft_consensus.cc:2804] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:10.059357 16309 raft_consensus.cc:697] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:10.060233 16309 consensus_queue.cc:237] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.072216 16010 catalog_manager.cc:5697] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 1 to 2, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 2 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.115775 16112 raft_consensus.cc:3060] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:10.212749 16334 raft_consensus.cc:993] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:10.213245 16340 raft_consensus.cc:1081] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Signalling peer 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:10.214766 16260 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:36432
I20260709 13:17:10.215283 16260 raft_consensus.cc:493] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.215582 16260 raft_consensus.cc:3060] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.220204 16260 raft_consensus.cc:515] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.221923 16260 leader_election.cc:290] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 2 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:10.223011 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:10.223026 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36a2dd202cb74d5890cbb655827d79a4" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:10.223719 16187 raft_consensus.cc:3060] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.223750 16112 raft_consensus.cc:3055] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:10.224288 16112 raft_consensus.cc:740] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:10.225189 16112 consensus_queue.cc:260] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.226480 16112 raft_consensus.cc:3060] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.231379 16187 raft_consensus.cc:2468] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 2.
I20260709 13:17:10.232574 16334 raft_consensus.cc:993] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:10.232656 16225 leader_election.cc:304] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:10.233119 16340 raft_consensus.cc:1081] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Signalling peer 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:10.233500 16307 raft_consensus.cc:2804] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:10.234433 16112 raft_consensus.cc:2468] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 2.
I20260709 13:17:10.234939 16187 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "725759fe33b34004ab1573af41da8c94"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:32970
I20260709 13:17:10.235445 16307 raft_consensus.cc:697] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Becoming Leader. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:10.235628 16187 raft_consensus.cc:493] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:10.236141 16187 raft_consensus.cc:3060] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.236361 16307 consensus_queue.cc:237] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.241717 16187 raft_consensus.cc:515] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.243631 16187 leader_election.cc:290] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 2 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:10.244323 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:10.244058 16009 catalog_manager.cc:5697] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 reported cstate change: term changed from 1 to 2, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3). New cstate: current_term: 2 leader_uuid: "3aaefafdc9a44a6691875b012f3defe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:10.244643 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "725759fe33b34004ab1573af41da8c94" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:10.244933 16112 raft_consensus.cc:3055] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:10.245262 16260 raft_consensus.cc:3060] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.245333 16112 raft_consensus.cc:740] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:10.246183 16112 consensus_queue.cc:260] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.247172 16112 raft_consensus.cc:3060] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.251224 16260 raft_consensus.cc:2468] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 2.
I20260709 13:17:10.252586 16153 leader_election.cc:304] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:10.252982 16112 raft_consensus.cc:2468] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 2.
I20260709 13:17:10.253516 16309 raft_consensus.cc:2804] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:10.254040 16309 raft_consensus.cc:697] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:10.254757 16309 consensus_queue.cc:237] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:10.260664 16009 catalog_manager.cc:5697] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 1 to 2, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 2 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:10.523780 16112 raft_consensus.cc:1275] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.525437 16309 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.529857 16260 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.532689 16261 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:10.537199 16260 raft_consensus.cc:1275] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.538568 16324 consensus_queue.cc:1048] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.540561 16261 raft_consensus.cc:1275] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.543365 16359 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.548043 16187 raft_consensus.cc:1275] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.549465 16341 consensus_queue.cc:1048] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.691577 16259 raft_consensus.cc:1275] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.692759 16359 consensus_queue.cc:1048] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.699939 16112 raft_consensus.cc:1275] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.701959 16308 consensus_queue.cc:1048] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.758381 16112 raft_consensus.cc:1275] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.759673 16307 consensus_queue.cc:1048] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:10.773216 16187 raft_consensus.cc:1275] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:10.774652 16395 consensus_queue.cc:1048] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:12.805768 15476 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:17:12.806463 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:12.823006 16112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:44652
I20260709 13:17:12.823510 16112 raft_consensus.cc:606] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Received request to transfer leadership to TS 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:12.825845 16112 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:44652
I20260709 13:17:12.826359 16112 raft_consensus.cc:606] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Received request to transfer leadership to TS 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:12.827499 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:12.827970 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:13.012367 16398 raft_consensus.cc:993] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:13.012835 16403 raft_consensus.cc:1081] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Signalling peer 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:13.014196 16260 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:36432
I20260709 13:17:13.014654 16260 raft_consensus.cc:493] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.014968 16260 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.019150 16260 raft_consensus.cc:515] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.020538 16260 leader_election.cc:290] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 2 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:13.021481 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:13.021598 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:13.022058 16112 raft_consensus.cc:3055] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:13.022198 16187 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.022372 16112 raft_consensus.cc:740] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:13.023160 16112 consensus_queue.cc:260] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.024060 16112 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.027536 16187 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 2.
I20260709 13:17:13.028770 16225 leader_election.cc:304] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:13.029392 16112 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 2.
I20260709 13:17:13.029563 16395 raft_consensus.cc:2804] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:13.030860 16395 raft_consensus.cc:697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Becoming Leader. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:13.031724 16395 consensus_queue.cc:237] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.039682 16010 catalog_manager.cc:5697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 reported cstate change: term changed from 1 to 2, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3). New cstate: current_term: 2 leader_uuid: "3aaefafdc9a44a6691875b012f3defe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.221827 16398 raft_consensus.cc:993] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:13.222301 16403 raft_consensus.cc:1081] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Signalling peer 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:13.223729 16259 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:36432
I20260709 13:17:13.224190 16259 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.224483 16259 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.230428 16259 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.232409 16259 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 2 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:13.233223 16112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:13.233352 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:13.233835 16112 raft_consensus.cc:3055] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:13.233902 16187 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.234236 16112 raft_consensus.cc:740] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:13.235070 16112 consensus_queue.cc:260] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.236253 16112 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.240475 16187 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 2.
I20260709 13:17:13.241524 16225 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:13.242272 16395 raft_consensus.cc:2804] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:13.242688 16395 raft_consensus.cc:697] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Becoming Leader. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:13.242779 16112 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 2.
I20260709 13:17:13.243505 16395 consensus_queue.cc:237] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.250285 16010 catalog_manager.cc:5697] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 reported cstate change: term changed from 1 to 2, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3). New cstate: current_term: 2 leader_uuid: "3aaefafdc9a44a6691875b012f3defe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:13.510679 16187 raft_consensus.cc:1275] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.511945 16395 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:13.526150 16112 raft_consensus.cc:1275] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.527648 16404 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:13.748471 16187 raft_consensus.cc:1275] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.749761 16413 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:13.757725 16111 raft_consensus.cc:1275] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:13.759419 16395 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:13.828830 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:13.853094 16259 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
mode: GRACEFUL
new_leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:36436
I20260709 13:17:13.853611 16259 raft_consensus.cc:606] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Received request to transfer leadership to TS 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:13.861491 16259 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
mode: GRACEFUL
new_leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:36436
I20260709 13:17:13.862007 16259 raft_consensus.cc:606] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Received request to transfer leadership to TS 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:13.867547 16187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
mode: GRACEFUL
new_leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:32974
I20260709 13:17:13.868201 16187 raft_consensus.cc:606] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Received request to transfer leadership to TS 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:13.874684 16187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
mode: GRACEFUL
new_leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:32974
I20260709 13:17:13.875409 16187 raft_consensus.cc:606] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Received request to transfer leadership to TS 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:13.881078 16187 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
mode: GRACEFUL
new_leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:32974
I20260709 13:17:13.881618 16187 raft_consensus.cc:606] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Received request to transfer leadership to TS 691fdf9ebed94402a4d88c99ea52d10e
I20260709 13:17:13.898666 16308 raft_consensus.cc:993] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: : Instructing follower 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:13.899170 16421 raft_consensus.cc:1081] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Signalling peer 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:13.900722 16111 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:44630
I20260709 13:17:13.901193 16111 raft_consensus.cc:493] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.901490 16111 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.905917 16111 raft_consensus.cc:515] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.907595 16111 leader_election.cc:290] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 3 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:13.908313 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:13.908676 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:13.908874 16187 raft_consensus.cc:3055] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:13.909227 16187 raft_consensus.cc:740] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:13.909192 16259 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.910040 16187 consensus_queue.cc:260] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.910912 16187 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.914301 16259 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 3.
I20260709 13:17:13.915194 16187 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 3.
I20260709 13:17:13.915505 16078 leader_election.cc:304] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:13.916307 16403 raft_consensus.cc:2804] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:13.917270 16308 raft_consensus.cc:993] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: : Instructing follower 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:13.917665 16403 raft_consensus.cc:697] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:13.917762 16421 raft_consensus.cc:1081] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Signalling peer 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:13.918495 16403 consensus_queue.cc:237] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.919443 16111 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:44630
I20260709 13:17:13.919922 16111 raft_consensus.cc:493] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.920256 16111 raft_consensus.cc:3060] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.926069 16111 raft_consensus.cc:515] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.926452 16009 catalog_manager.cc:5697] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 2 to 3, leader changed from 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2) to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 3 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.928290 16111 leader_election.cc:290] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 2 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:13.929526 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:13.930219 16259 raft_consensus.cc:3060] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.935449 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d2b63f8158d341beb37aa11394c9b562" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:13.936632 16259 raft_consensus.cc:2468] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 2.
I20260709 13:17:13.937032 16187 raft_consensus.cc:3055] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:13.937381 16187 raft_consensus.cc:740] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:13.937937 16078 leader_election.cc:304] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:13.938061 16187 consensus_queue.cc:260] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.939263 16187 raft_consensus.cc:3060] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:13.940236 16403 raft_consensus.cc:2804] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:13.940681 16403 raft_consensus.cc:697] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:13.941449 16403 consensus_queue.cc:237] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:13.944048 16187 raft_consensus.cc:2468] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 2.
I20260709 13:17:13.948552 16009 catalog_manager.cc:5697] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 1 to 2, leader changed from 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2) to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:13.992108 16404 raft_consensus.cc:993] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: : Instructing follower 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:13.992568 16395 raft_consensus.cc:1081] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Signalling peer 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:13.994177 16111 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:44636
I20260709 13:17:13.994743 16111 raft_consensus.cc:493] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:13.995139 16111 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:13.999925 16111 raft_consensus.cc:515] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.001641 16111 leader_election.cc:290] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 3 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:14.002812 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:14.002841 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:14.003539 16187 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.003633 16259 raft_consensus.cc:3055] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:14.003986 16259 raft_consensus.cc:740] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:14.004838 16259 consensus_queue.cc:260] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.005762 16259 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.008365 16187 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 3.
I20260709 13:17:14.009424 16076 leader_election.cc:304] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:14.010205 16403 raft_consensus.cc:2804] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:14.010607 16403 raft_consensus.cc:697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:14.011066 16259 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 3.
I20260709 13:17:14.011399 16403 consensus_queue.cc:237] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.017573 16009 catalog_manager.cc:5697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 2 to 3, leader changed from 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3) to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 3 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.142818 16308 raft_consensus.cc:993] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: : Instructing follower 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:14.143420 16421 raft_consensus.cc:1081] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Signalling peer 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:14.145364 16111 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:44630
I20260709 13:17:14.146049 16111 raft_consensus.cc:493] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.146517 16111 raft_consensus.cc:3060] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.155485 16111 raft_consensus.cc:515] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.157965 16111 leader_election.cc:290] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 2 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:14.158717 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:14.159171 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc5cdb12fb3a4c58ad6399da4179ac36" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:14.159351 16187 raft_consensus.cc:3055] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:14.159691 16259 raft_consensus.cc:3060] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.159669 16187 raft_consensus.cc:740] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:14.160490 16187 consensus_queue.cc:260] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.161340 16187 raft_consensus.cc:3060] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:14.164920 16259 raft_consensus.cc:2468] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 2.
I20260709 13:17:14.166028 16078 leader_election.cc:304] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:14.165437 16187 raft_consensus.cc:2468] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 2.
I20260709 13:17:14.166733 16403 raft_consensus.cc:2804] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:14.167258 16403 raft_consensus.cc:697] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:14.167949 16403 consensus_queue.cc:237] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.174306 16009 catalog_manager.cc:5697] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 1 to 2, leader changed from 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2) to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 2 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.294996 16413 raft_consensus.cc:993] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: : Instructing follower 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:14.295431 16395 raft_consensus.cc:1081] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Signalling peer 691fdf9ebed94402a4d88c99ea52d10e to start an election
I20260709 13:17:14.297012 16111 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
 from {username='slave'} at 127.0.0.1:44636
I20260709 13:17:14.297542 16111 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:14.297880 16111 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.302510 16111 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.304334 16111 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 3 election: Requested vote from peers 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:14.305152 16187 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:14.305460 16259 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "691fdf9ebed94402a4d88c99ea52d10e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:14.305733 16187 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.305976 16259 raft_consensus.cc:3055] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:14.306284 16259 raft_consensus.cc:740] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:14.307071 16259 consensus_queue.cc:260] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.308218 16259 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:14.312014 16187 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 3.
I20260709 13:17:14.313138 16076 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d77c04a96e4404c88186359f9ec6bdd, 691fdf9ebed94402a4d88c99ea52d10e; no voters: 
I20260709 13:17:14.313915 16403 raft_consensus.cc:2804] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:14.314318 16403 raft_consensus.cc:697] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Becoming Leader. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:14.314695 16259 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 691fdf9ebed94402a4d88c99ea52d10e in term 3.
I20260709 13:17:14.315141 16403 consensus_queue.cc:237] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:14.320690 16187 raft_consensus.cc:1275] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.321292 16009 catalog_manager.cc:5697] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e reported cstate change: term changed from 2 to 3, leader changed from 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3) to 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1). New cstate: current_term: 3 leader_uuid: "691fdf9ebed94402a4d88c99ea52d10e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:14.321960 16403 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:14.332703 16259 raft_consensus.cc:1275] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.334408 16403 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:14.343665 16260 raft_consensus.cc:1275] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.344848 16403 consensus_queue.cc:1048] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.357779 16187 raft_consensus.cc:1275] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.359328 16422 consensus_queue.cc:1048] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.445231 16186 raft_consensus.cc:1275] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.446592 16422 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:14.454968 16260 raft_consensus.cc:1275] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.456250 16422 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:14.573640 16260 raft_consensus.cc:1275] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.574900 16422 consensus_queue.cc:1048] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.583024 16186 raft_consensus.cc:1275] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:14.584421 16403 consensus_queue.cc:1048] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:14.736073 16186 raft_consensus.cc:1275] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.737484 16403 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:14.745323 16260 raft_consensus.cc:1275] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Refusing update from remote peer 691fdf9ebed94402a4d88c99ea52d10e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:14.747121 16439 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:16.889792 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:16.897390 16111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:44652
I20260709 13:17:16.898154 16111 raft_consensus.cc:606] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Received request to transfer leadership to TS 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:16.900863 16111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:44652
I20260709 13:17:16.901405 16111 raft_consensus.cc:606] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Received request to transfer leadership to TS 3aaefafdc9a44a6691875b012f3defe1
I20260709 13:17:16.903688 16111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:44652
I20260709 13:17:16.904209 16111 raft_consensus.cc:606] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Received request to transfer leadership to TS 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:16.906265 16111 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6"
dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
mode: GRACEFUL
new_leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:44652
I20260709 13:17:16.906713 16111 raft_consensus.cc:606] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Received request to transfer leadership to TS 4d77c04a96e4404c88186359f9ec6bdd
I20260709 13:17:16.907851 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:17:16.908258 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:17.235272 16442 raft_consensus.cc:993] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:17.235942 16441 raft_consensus.cc:1081] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Signalling peer 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:17.237651 16186 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:32970
I20260709 13:17:17.238211 16186 raft_consensus.cc:493] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.238552 16186 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.244089 16186 raft_consensus.cc:515] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.245831 16186 leader_election.cc:290] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 4 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:17.246752 16111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:17.247262 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a244ab737d95446a8f2563ced8410f01" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:17.247494 16111 raft_consensus.cc:3055] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:17.247763 16260 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.247813 16111 raft_consensus.cc:740] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:17.248721 16111 consensus_queue.cc:260] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.249774 16111 raft_consensus.cc:3060] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.253005 16260 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 4.
I20260709 13:17:17.254207 16153 leader_election.cc:304] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:17.255028 16421 raft_consensus.cc:2804] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:17.255226 16111 raft_consensus.cc:2468] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 4.
I20260709 13:17:17.256026 16421 raft_consensus.cc:697] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:17.257200 16421 consensus_queue.cc:237] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.263870 16009 catalog_manager.cc:5697] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 3 to 4, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 4 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:17.280671 16442 raft_consensus.cc:993] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:17.281132 16441 raft_consensus.cc:1081] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Signalling peer 4d77c04a96e4404c88186359f9ec6bdd to start an election
I20260709 13:17:17.282581 16186 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6"
dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
 from {username='slave'} at 127.0.0.1:32970
I20260709 13:17:17.283139 16186 raft_consensus.cc:493] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.283465 16186 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.289214 16186 raft_consensus.cc:515] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.291165 16186 leader_election.cc:290] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 3 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3:38939)
I20260709 13:17:17.292016 16111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:17.292178 16260 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6371667f79a40f688f1abc3a81506b6" candidate_uuid: "4d77c04a96e4404c88186359f9ec6bdd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
I20260709 13:17:17.292567 16111 raft_consensus.cc:3055] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:17.292690 16260 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.292891 16111 raft_consensus.cc:740] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:17.293711 16111 consensus_queue.cc:260] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.294617 16111 raft_consensus.cc:3060] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:17.297268 16260 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 3.
I20260709 13:17:17.298179 16153 leader_election.cc:304] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:17.298882 16421 raft_consensus.cc:2804] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:17.299350 16421 raft_consensus.cc:697] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 LEADER]: Becoming Leader. State: Replica: 4d77c04a96e4404c88186359f9ec6bdd, State: Running, Role: LEADER
I20260709 13:17:17.299850 16111 raft_consensus.cc:2468] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d77c04a96e4404c88186359f9ec6bdd in term 3.
I20260709 13:17:17.300271 16421 consensus_queue.cc:237] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.306792 16009 catalog_manager.cc:5697] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd reported cstate change: term changed from 2 to 3, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2). New cstate: current_term: 3 leader_uuid: "4d77c04a96e4404c88186359f9ec6bdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:17.340198 16442 raft_consensus.cc:993] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:17.340655 16441 raft_consensus.cc:1081] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Signalling peer 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:17.342100 16260 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:36432
I20260709 13:17:17.342506 16260 raft_consensus.cc:493] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.342733 16260 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.347077 16260 raft_consensus.cc:515] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.348667 16260 leader_election.cc:290] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 4 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:17.349459 16111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:17.349762 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b3a442dbdc4742a3684d96cb8d7cbd" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:17.350041 16111 raft_consensus.cc:3055] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:17.350376 16186 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.350402 16111 raft_consensus.cc:740] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:17.351310 16111 consensus_queue.cc:260] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.352231 16111 raft_consensus.cc:3060] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.355660 16186 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 4.
I20260709 13:17:17.356969 16111 raft_consensus.cc:2468] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 4.
I20260709 13:17:17.357062 16225 leader_election.cc:304] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:17.358086 16395 raft_consensus.cc:2804] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:17.359241 16395 raft_consensus.cc:697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 4 LEADER]: Becoming Leader. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:17.360136 16395 consensus_queue.cc:237] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.367383 16009 catalog_manager.cc:5697] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 reported cstate change: term changed from 3 to 4, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3). New cstate: current_term: 4 leader_uuid: "3aaefafdc9a44a6691875b012f3defe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.437640 16442 raft_consensus.cc:993] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: : Instructing follower 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:17.438071 16441 raft_consensus.cc:1081] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Signalling peer 3aaefafdc9a44a6691875b012f3defe1 to start an election
I20260709 13:17:17.439343 16260 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9"
dest_uuid: "3aaefafdc9a44a6691875b012f3defe1"
 from {username='slave'} at 127.0.0.1:36432
I20260709 13:17:17.439782 16260 raft_consensus.cc:493] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:17.440063 16260 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.444199 16260 raft_consensus.cc:515] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.445710 16260 leader_election.cc:290] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 4 election: Requested vote from peers 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529), 4d77c04a96e4404c88186359f9ec6bdd (127.15.29.2:38899)
I20260709 13:17:17.446445 16111 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "691fdf9ebed94402a4d88c99ea52d10e"
I20260709 13:17:17.446780 16186 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34c39404ff264334a86dbfba157d1fd9" candidate_uuid: "3aaefafdc9a44a6691875b012f3defe1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4d77c04a96e4404c88186359f9ec6bdd"
I20260709 13:17:17.447012 16111 raft_consensus.cc:3055] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:17.447304 16186 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.447367 16111 raft_consensus.cc:740] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 691fdf9ebed94402a4d88c99ea52d10e, State: Running, Role: LEADER
I20260709 13:17:17.448247 16111 consensus_queue.cc:260] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.449175 16111 raft_consensus.cc:3060] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:17.452256 16186 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 4.
I20260709 13:17:17.453228 16225 leader_election.cc:304] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaefafdc9a44a6691875b012f3defe1, 4d77c04a96e4404c88186359f9ec6bdd; no voters: 
I20260709 13:17:17.453938 16395 raft_consensus.cc:2804] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:17.454098 16111 raft_consensus.cc:2468] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3aaefafdc9a44a6691875b012f3defe1 in term 4.
I20260709 13:17:17.454419 16395 raft_consensus.cc:697] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 LEADER]: Becoming Leader. State: Replica: 3aaefafdc9a44a6691875b012f3defe1, State: Running, Role: LEADER
I20260709 13:17:17.455346 16395 consensus_queue.cc:237] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } }
I20260709 13:17:17.462272 16009 catalog_manager.cc:5697] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 reported cstate change: term changed from 3 to 4, leader changed from 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1) to 3aaefafdc9a44a6691875b012f3defe1 (127.15.29.3). New cstate: current_term: 4 leader_uuid: "3aaefafdc9a44a6691875b012f3defe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:17.821460 16260 raft_consensus.cc:1275] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:17.822706 16421 consensus_queue.cc:1048] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:17.835301 16111 raft_consensus.cc:1275] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:17.838433 16456 consensus_queue.cc:1048] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:17.850176 16111 raft_consensus.cc:1275] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.851320 16456 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:17.856637 16186 raft_consensus.cc:1275] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.857877 16395 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:17.861474 16260 raft_consensus.cc:1275] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Refusing update from remote peer 4d77c04a96e4404c88186359f9ec6bdd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.862694 16421 consensus_queue.cc:1048] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaefafdc9a44a6691875b012f3defe1" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 38939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:17.882927 16111 raft_consensus.cc:1275] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.884399 16463 consensus_queue.cc:1048] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:17.898440 16186 raft_consensus.cc:1275] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.899724 16463 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d77c04a96e4404c88186359f9ec6bdd" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 38899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:17.908177 16111 raft_consensus.cc:1275] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Refusing update from remote peer 3aaefafdc9a44a6691875b012f3defe1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:17.909061 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:17.912177 16463 consensus_queue.cc:1048] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "691fdf9ebed94402a4d88c99ea52d10e" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 40529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:17.919309 15476 tablet_server.cc:179] TabletServer@127.15.29.1:0 shutting down...
I20260709 13:17:17.947415 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:17.948112 15476 tablet_replica.cc:333] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.948686 15476 raft_consensus.cc:2243] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:17.949570 15476 raft_consensus.cc:2272] T 70ddb69735cc469db6aae77c64705301 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.952113 15476 tablet_replica.cc:333] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.953069 15476 raft_consensus.cc:2243] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:17.953848 15476 raft_consensus.cc:2272] T fc5cdb12fb3a4c58ad6399da4179ac36 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.956185 15476 tablet_replica.cc:333] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.956722 15476 raft_consensus.cc:2243] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.957139 15476 raft_consensus.cc:2272] T a244ab737d95446a8f2563ced8410f01 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.959265 15476 tablet_replica.cc:333] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.959808 15476 raft_consensus.cc:2243] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.960208 15476 raft_consensus.cc:2272] T d6371667f79a40f688f1abc3a81506b6 P 691fdf9ebed94402a4d88c99ea52d10e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.962255 15476 tablet_replica.cc:333] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.962744 15476 raft_consensus.cc:2243] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:17.963521 15476 raft_consensus.cc:2272] T d2b63f8158d341beb37aa11394c9b562 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.965178 15476 tablet_replica.cc:333] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.965684 15476 raft_consensus.cc:2243] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.966145 15476 raft_consensus.cc:2272] T 725759fe33b34004ab1573af41da8c94 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.968036 15476 tablet_replica.cc:333] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.968542 15476 raft_consensus.cc:2243] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.968943 15476 raft_consensus.cc:2272] T 34c39404ff264334a86dbfba157d1fd9 P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.971529 15476 tablet_replica.cc:333] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.972292 15476 raft_consensus.cc:2243] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.972821 15476 raft_consensus.cc:2272] T 59b3a442dbdc4742a3684d96cb8d7cbd P 691fdf9ebed94402a4d88c99ea52d10e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.975394 15476 tablet_replica.cc:333] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.975855 15476 raft_consensus.cc:2243] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:17.976718 15476 raft_consensus.cc:2272] T f3f2358cfe974d0a8ef79e0473ccb918 P 691fdf9ebed94402a4d88c99ea52d10e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.978927 15476 tablet_replica.cc:333] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e: stopping tablet replica
I20260709 13:17:17.979382 15476 raft_consensus.cc:2243] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.979782 15476 raft_consensus.cc:2272] T 36a2dd202cb74d5890cbb655827d79a4 P 691fdf9ebed94402a4d88c99ea52d10e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.009119 15476 tablet_server.cc:196] TabletServer@127.15.29.1:0 shutdown complete.
I20260709 13:17:18.031071 15476 tablet_server.cc:179] TabletServer@127.15.29.2:0 shutting down...
I20260709 13:17:18.056479 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:18.057173 15476 tablet_replica.cc:333] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.057754 15476 raft_consensus.cc:2243] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.058223 15476 raft_consensus.cc:2272] T 36a2dd202cb74d5890cbb655827d79a4 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.060127 15476 tablet_replica.cc:333] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.060688 15476 raft_consensus.cc:2243] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.061415 15476 raft_consensus.cc:2272] T d6371667f79a40f688f1abc3a81506b6 P 4d77c04a96e4404c88186359f9ec6bdd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.063753 15476 tablet_replica.cc:333] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.064620 15476 raft_consensus.cc:2243] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.065171 15476 raft_consensus.cc:2272] T f3f2358cfe974d0a8ef79e0473ccb918 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.066033 16227 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.29.1:40529: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260709 13:17:18.067636 15476 tablet_replica.cc:333] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.068430 15476 raft_consensus.cc:2243] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.068925 15476 raft_consensus.cc:2272] T 34c39404ff264334a86dbfba157d1fd9 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.069569 16227 consensus_peers.cc:597] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 -> Peer 691fdf9ebed94402a4d88c99ea52d10e (127.15.29.1:40529): Couldn't send request to peer 691fdf9ebed94402a4d88c99ea52d10e. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:40529: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.071734 15476 tablet_replica.cc:333] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.072280 15476 raft_consensus.cc:2243] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.072783 15476 raft_consensus.cc:2272] T 59b3a442dbdc4742a3684d96cb8d7cbd P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.074824 15476 tablet_replica.cc:333] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.075371 15476 raft_consensus.cc:2243] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.075776 15476 raft_consensus.cc:2272] T 70ddb69735cc469db6aae77c64705301 P 4d77c04a96e4404c88186359f9ec6bdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.077467 15476 tablet_replica.cc:333] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.077970 15476 raft_consensus.cc:2243] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.078423 15476 raft_consensus.cc:2272] T d2b63f8158d341beb37aa11394c9b562 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.080099 15476 tablet_replica.cc:333] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.080559 15476 raft_consensus.cc:2243] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.081240 15476 raft_consensus.cc:2272] T 725759fe33b34004ab1573af41da8c94 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.082849 15476 tablet_replica.cc:333] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.083348 15476 raft_consensus.cc:2243] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.084009 15476 raft_consensus.cc:2272] T a244ab737d95446a8f2563ced8410f01 P 4d77c04a96e4404c88186359f9ec6bdd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.085855 15476 tablet_replica.cc:333] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd: stopping tablet replica
I20260709 13:17:18.086318 15476 raft_consensus.cc:2243] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.086730 15476 raft_consensus.cc:2272] T fc5cdb12fb3a4c58ad6399da4179ac36 P 4d77c04a96e4404c88186359f9ec6bdd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.105954 15476 tablet_server.cc:196] TabletServer@127.15.29.2:0 shutdown complete.
I20260709 13:17:18.124316 15476 tablet_server.cc:179] TabletServer@127.15.29.3:0 shutting down...
I20260709 13:17:18.147176 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:18.147943 15476 tablet_replica.cc:333] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.148882 15476 raft_consensus.cc:2243] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.149381 15476 raft_consensus.cc:2272] T 70ddb69735cc469db6aae77c64705301 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.151238 15476 tablet_replica.cc:333] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.151749 15476 raft_consensus.cc:2243] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.152170 15476 raft_consensus.cc:2272] T d6371667f79a40f688f1abc3a81506b6 P 3aaefafdc9a44a6691875b012f3defe1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.154220 15476 tablet_replica.cc:333] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.154690 15476 raft_consensus.cc:2243] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.155449 15476 raft_consensus.cc:2272] T 36a2dd202cb74d5890cbb655827d79a4 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.157110 15476 tablet_replica.cc:333] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.157577 15476 raft_consensus.cc:2243] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.157974 15476 raft_consensus.cc:2272] T 725759fe33b34004ab1573af41da8c94 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.159731 15476 tablet_replica.cc:333] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.160166 15476 raft_consensus.cc:2243] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.160532 15476 raft_consensus.cc:2272] T d2b63f8158d341beb37aa11394c9b562 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.162132 15476 tablet_replica.cc:333] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.162559 15476 raft_consensus.cc:2243] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.163280 15476 raft_consensus.cc:2272] T 59b3a442dbdc4742a3684d96cb8d7cbd P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.165117 15476 tablet_replica.cc:333] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.165542 15476 raft_consensus.cc:2243] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.166239 15476 raft_consensus.cc:2272] T 34c39404ff264334a86dbfba157d1fd9 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.168095 15476 tablet_replica.cc:333] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.168531 15476 raft_consensus.cc:2243] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.168884 15476 raft_consensus.cc:2272] T f3f2358cfe974d0a8ef79e0473ccb918 P 3aaefafdc9a44a6691875b012f3defe1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.170473 15476 tablet_replica.cc:333] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.171012 15476 raft_consensus.cc:2243] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.171371 15476 raft_consensus.cc:2272] T a244ab737d95446a8f2563ced8410f01 P 3aaefafdc9a44a6691875b012f3defe1 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.173159 15476 tablet_replica.cc:333] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1: stopping tablet replica
I20260709 13:17:18.173550 15476 raft_consensus.cc:2243] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.173913 15476 raft_consensus.cc:2272] T fc5cdb12fb3a4c58ad6399da4179ac36 P 3aaefafdc9a44a6691875b012f3defe1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.201092 15476 tablet_server.cc:196] TabletServer@127.15.29.3:0 shutdown complete.
I20260709 13:17:18.219383 15476 master.cc:562] Master@127.15.29.62:33389 shutting down...
I20260709 13:17:18.239338 15476 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.239907 15476 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.240257 15476 tablet_replica.cc:333] T 00000000000000000000000000000000 P 572bfa6ecd224de29c4d5ea51babff36: stopping tablet replica
I20260709 13:17:18.260531 15476 master.cc:584] Master@127.15.29.62:33389 shutdown complete.
3aaefafdc9a44a6691875b012f3defe1  1
4d77c04a96e4404c88186359f9ec6bdd  4
691fdf9ebed94402a4d88c99ea52d10e  5
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10403 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:17:18.302048 15476 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.29.62:37247
I20260709 13:17:18.303301 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.309337 16469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.309358 16470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.310441 15476 server_base.cc:1061] running on GCE node
W20260709 13:17:18.310402 16472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.311601 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.311792 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.311909 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603038311898 us; error 0 us; skew 500 ppm
I20260709 13:17:18.312357 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.314772 15476 webserver.cc:533] Webserver started at http://127.15.29.62:43965/ using document root <none> and password file <none>
I20260709 13:17:18.315343 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.315507 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.315739 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.316891 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/master-0-root/instance:
uuid: "bfe27eaae9f4449e96031a0b1485f80c"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.321611 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:17:18.325148 16477 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.326007 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:18.326330 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/master-0-root
uuid: "bfe27eaae9f4449e96031a0b1485f80c"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.326647 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.346197 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.347656 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.387305 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.62:37247
I20260709 13:17:18.387382 16528 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.62:37247 every 8 connection(s)
I20260709 13:17:18.391090 16529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:18.400831 16529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Bootstrap starting.
I20260709 13:17:18.405253 16529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:18.409256 16529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: No bootstrap required, opened a new log
I20260709 13:17:18.411300 16529 raft_consensus.cc:359] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfe27eaae9f4449e96031a0b1485f80c" member_type: VOTER }
I20260709 13:17:18.411659 16529 raft_consensus.cc:385] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:18.411834 16529 raft_consensus.cc:740] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfe27eaae9f4449e96031a0b1485f80c, State: Initialized, Role: FOLLOWER
I20260709 13:17:18.412395 16529 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfe27eaae9f4449e96031a0b1485f80c" member_type: VOTER }
I20260709 13:17:18.412801 16529 raft_consensus.cc:399] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:18.412984 16529 raft_consensus.cc:493] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:18.413184 16529 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:18.417444 16529 raft_consensus.cc:515] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfe27eaae9f4449e96031a0b1485f80c" member_type: VOTER }
I20260709 13:17:18.417992 16529 leader_election.cc:304] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfe27eaae9f4449e96031a0b1485f80c; no voters: 
I20260709 13:17:18.419076 16529 leader_election.cc:290] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:18.419471 16532 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:18.420703 16532 raft_consensus.cc:697] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [term 1 LEADER]: Becoming Leader. State: Replica: bfe27eaae9f4449e96031a0b1485f80c, State: Running, Role: LEADER
I20260709 13:17:18.421360 16532 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfe27eaae9f4449e96031a0b1485f80c" member_type: VOTER }
I20260709 13:17:18.422020 16529 sys_catalog.cc:565] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:18.426290 16533 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfe27eaae9f4449e96031a0b1485f80c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfe27eaae9f4449e96031a0b1485f80c" member_type: VOTER } }
I20260709 13:17:18.426723 16534 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfe27eaae9f4449e96031a0b1485f80c. Latest consensus state: current_term: 1 leader_uuid: "bfe27eaae9f4449e96031a0b1485f80c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfe27eaae9f4449e96031a0b1485f80c" member_type: VOTER } }
I20260709 13:17:18.427183 16533 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:18.427568 16534 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:18.430585 16539 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:18.435770 16539 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:18.440068 15476 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:18.444631 16539 catalog_manager.cc:1383] Generated new cluster ID: 500d471ede0546f2bb1eed2cc5f1e0c0
I20260709 13:17:18.444928 16539 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:18.465282 16539 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:18.466605 16539 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:18.478672 16539 catalog_manager.cc:6070] T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Generated new TSK 0
I20260709 13:17:18.479332 16539 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:18.506441 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.512042 16550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.513208 16551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.515024 16553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.515190 15476 server_base.cc:1061] running on GCE node
I20260709 13:17:18.516435 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.516607 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.516831 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603038516812 us; error 0 us; skew 500 ppm
I20260709 13:17:18.517378 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.519544 15476 webserver.cc:533] Webserver started at http://127.15.29.1:40451/ using document root <none> and password file <none>
I20260709 13:17:18.520004 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.520198 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.520467 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.521479 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/instance:
uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.525961 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:17:18.529187 16558 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.529953 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:18.530256 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root
uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.530532 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.547550 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.548854 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.550398 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:18.552762 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:18.552937 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.553258 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:18.553434 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.593088 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.1:34889
I20260709 13:17:18.593206 16620 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.1:34889 every 8 connection(s)
I20260709 13:17:18.598243 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.605304 16625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.605762 16626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.608436 15476 server_base.cc:1061] running on GCE node
W20260709 13:17:18.609186 16629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.610158 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.610342 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.610463 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603038610452 us; error 0 us; skew 500 ppm
I20260709 13:17:18.610982 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.613762 15476 webserver.cc:533] Webserver started at http://127.15.29.2:39163/ using document root <none> and password file <none>
I20260709 13:17:18.614346 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.614696 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.615047 16621 heartbeater.cc:344] Connected to a master server at 127.15.29.62:37247
I20260709 13:17:18.615048 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.615563 16621 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:18.616303 16621 heartbeater.cc:507] Master 127.15.29.62:37247 requested a full tablet report, sending...
I20260709 13:17:18.616649 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/instance:
uuid: "170921ba08b74a33995ae7c7686f8109"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.618391 16494 ts_manager.cc:194] Registered new tserver with Master: 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:18.620196 16494 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38206
I20260709 13:17:18.621724 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:18.625433 16633 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.626257 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:17:18.626588 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root
uuid: "170921ba08b74a33995ae7c7686f8109"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.626941 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.650797 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.652055 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.653484 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:18.655622 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:18.655843 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.656087 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:18.656270 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.695956 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.2:42639
I20260709 13:17:18.696058 16695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.2:42639 every 8 connection(s)
I20260709 13:17:18.700716 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:18.707542 16699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:18.714201 16700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.718535 16696 heartbeater.cc:344] Connected to a master server at 127.15.29.62:37247
I20260709 13:17:18.719065 16696 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:18.720043 16696 heartbeater.cc:507] Master 127.15.29.62:37247 requested a full tablet report, sending...
I20260709 13:17:18.720117 15476 server_base.cc:1061] running on GCE node
W20260709 13:17:18.720613 16702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:18.721526 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:18.721751 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:18.721939 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603038721921 us; error 0 us; skew 500 ppm
I20260709 13:17:18.722323 16494 ts_manager.cc:194] Registered new tserver with Master: 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:18.722434 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:18.723865 16494 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38208
I20260709 13:17:18.725088 15476 webserver.cc:533] Webserver started at http://127.15.29.3:44649/ using document root <none> and password file <none>
I20260709 13:17:18.725704 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:18.725962 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:18.726277 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:18.727382 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/instance:
uuid: "efa00eb5e7404570958fdebb6cd077e8"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.731861 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:17:18.735049 16707 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.735788 15476 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:17:18.736083 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root
uuid: "efa00eb5e7404570958fdebb6cd077e8"
format_stamp: "Formatted at 2026-07-09 13:17:18 on dist-test-slave-hz8m"
I20260709 13:17:18.736364 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:18.754202 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:18.755554 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:18.757086 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:18.759572 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:18.759773 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.760078 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:18.760255 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:18.800230 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.3:36655
I20260709 13:17:18.800331 16769 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.3:36655 every 8 connection(s)
I20260709 13:17:18.814116 16770 heartbeater.cc:344] Connected to a master server at 127.15.29.62:37247
I20260709 13:17:18.814524 16770 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:18.815416 16770 heartbeater.cc:507] Master 127.15.29.62:37247 requested a full tablet report, sending...
I20260709 13:17:18.817394 16494 ts_manager.cc:194] Registered new tserver with Master: efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:18.818194 15476 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014626775s
I20260709 13:17:18.819087 16494 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38212
I20260709 13:17:18.845343 16494 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38216:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:18.851444 16494 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:19.047963 16735 tablet_service.cc:1511] Processing CreateTablet for tablet bb360c38974b4352a4febd1e0aada45d (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:19.048596 16734 tablet_service.cc:1511] Processing CreateTablet for tablet d9aa91533ea94ae392057ef582e0c419 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:19.049513 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb360c38974b4352a4febd1e0aada45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.050168 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9aa91533ea94ae392057ef582e0c419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.054100 16733 tablet_service.cc:1511] Processing CreateTablet for tablet 3249701d37834e80bbe408e24ced7d54 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:19.055546 16733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3249701d37834e80bbe408e24ced7d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.062762 16586 tablet_service.cc:1511] Processing CreateTablet for tablet bb360c38974b4352a4febd1e0aada45d (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:19.064280 16586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb360c38974b4352a4febd1e0aada45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.063930 16583 tablet_service.cc:1511] Processing CreateTablet for tablet 2125e4b55933488bbb6e25dba4189adb (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:19.065426 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2125e4b55933488bbb6e25dba4189adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.065806 16731 tablet_service.cc:1511] Processing CreateTablet for tablet f71532fa5e704f9590387cd12f98666b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:19.066416 16585 tablet_service.cc:1511] Processing CreateTablet for tablet d9aa91533ea94ae392057ef582e0c419 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:19.067217 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71532fa5e704f9590387cd12f98666b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.067808 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9aa91533ea94ae392057ef582e0c419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.071161 16732 tablet_service.cc:1511] Processing CreateTablet for tablet 2125e4b55933488bbb6e25dba4189adb (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:19.072230 16584 tablet_service.cc:1511] Processing CreateTablet for tablet 3249701d37834e80bbe408e24ced7d54 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:19.072726 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2125e4b55933488bbb6e25dba4189adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.073127 16582 tablet_service.cc:1511] Processing CreateTablet for tablet f71532fa5e704f9590387cd12f98666b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:19.073603 16584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3249701d37834e80bbe408e24ced7d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.074411 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71532fa5e704f9590387cd12f98666b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.077695 16730 tablet_service.cc:1511] Processing CreateTablet for tablet cc977d4c44af4e80b58e4e0864085d9a (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:19.080039 16729 tablet_service.cc:1511] Processing CreateTablet for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:19.081149 16580 tablet_service.cc:1511] Processing CreateTablet for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:19.081622 16581 tablet_service.cc:1511] Processing CreateTablet for tablet cc977d4c44af4e80b58e4e0864085d9a (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:19.082544 16580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.083247 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc977d4c44af4e80b58e4e0864085d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.083294 16728 tablet_service.cc:1511] Processing CreateTablet for tablet 8243711a48fd4d8c85d79e7c10801c4f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:19.084793 16730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc977d4c44af4e80b58e4e0864085d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.084640 16579 tablet_service.cc:1511] Processing CreateTablet for tablet 8243711a48fd4d8c85d79e7c10801c4f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:19.085935 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8243711a48fd4d8c85d79e7c10801c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.086984 16728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8243711a48fd4d8c85d79e7c10801c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.091245 16729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.091985 16578 tablet_service.cc:1511] Processing CreateTablet for tablet d02ec9cc775248dfa8321ae141d0ad37 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:19.093281 16578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02ec9cc775248dfa8321ae141d0ad37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.099506 16577 tablet_service.cc:1511] Processing CreateTablet for tablet 9b74fd3415d14ae3aeb38c349eca0494 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:19.100986 16577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74fd3415d14ae3aeb38c349eca0494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.101017 16727 tablet_service.cc:1511] Processing CreateTablet for tablet d02ec9cc775248dfa8321ae141d0ad37 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:19.104820 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02ec9cc775248dfa8321ae141d0ad37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.111681 16661 tablet_service.cc:1511] Processing CreateTablet for tablet bb360c38974b4352a4febd1e0aada45d (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:17:19.113345 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb360c38974b4352a4febd1e0aada45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.118492 16654 tablet_service.cc:1511] Processing CreateTablet for tablet 8243711a48fd4d8c85d79e7c10801c4f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:17:19.118492 16659 tablet_service.cc:1511] Processing CreateTablet for tablet 3249701d37834e80bbe408e24ced7d54 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:17:19.119539 16657 tablet_service.cc:1511] Processing CreateTablet for tablet f71532fa5e704f9590387cd12f98666b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:17:19.118544 16656 tablet_service.cc:1511] Processing CreateTablet for tablet cc977d4c44af4e80b58e4e0864085d9a (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:17:19.121990 16655 tablet_service.cc:1511] Processing CreateTablet for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:17:19.124341 16660 tablet_service.cc:1511] Processing CreateTablet for tablet d9aa91533ea94ae392057ef582e0c419 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:17:19.127724 16653 tablet_service.cc:1511] Processing CreateTablet for tablet d02ec9cc775248dfa8321ae141d0ad37 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:17:19.124341 16658 tablet_service.cc:1511] Processing CreateTablet for tablet 2125e4b55933488bbb6e25dba4189adb (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:17:19.130103 16726 tablet_service.cc:1511] Processing CreateTablet for tablet 9b74fd3415d14ae3aeb38c349eca0494 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:19.126814 16654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8243711a48fd4d8c85d79e7c10801c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.132851 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74fd3415d14ae3aeb38c349eca0494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.134707 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02ec9cc775248dfa8321ae141d0ad37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.138993 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2125e4b55933488bbb6e25dba4189adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.133404 16652 tablet_service.cc:1511] Processing CreateTablet for tablet 9b74fd3415d14ae3aeb38c349eca0494 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:17:19.141646 16652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74fd3415d14ae3aeb38c349eca0494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.143748 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9aa91533ea94ae392057ef582e0c419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.145737 16657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71532fa5e704f9590387cd12f98666b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.147131 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3249701d37834e80bbe408e24ced7d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.148512 16656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc977d4c44af4e80b58e4e0864085d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.149920 16655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.165354 16729 tablet_service.cc:1511] Processing CreateTablet for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:19.170467 16581 tablet_service.cc:1511] Processing CreateTablet for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:19.171883 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.172470 16582 tablet_service.cc:1511] Processing CreateTablet for tablet ee99829f455541f09dac21015b00282e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:19.173758 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee99829f455541f09dac21015b00282e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.177307 16585 tablet_service.cc:1511] Processing CreateTablet for tablet 8b778f817264440987b35333e6540c72 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:19.180487 16734 tablet_service.cc:1511] Processing CreateTablet for tablet ee99829f455541f09dac21015b00282e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:19.178728 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b778f817264440987b35333e6540c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.208451 16583 tablet_service.cc:1511] Processing CreateTablet for tablet 3326ab1787714e708abfea6cf0509109 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:19.220904 16729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.223076 16727 tablet_service.cc:1511] Processing CreateTablet for tablet 8b778f817264440987b35333e6540c72 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:19.228807 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b778f817264440987b35333e6540c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.238297 16732 tablet_service.cc:1511] Processing CreateTablet for tablet 3326ab1787714e708abfea6cf0509109 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:19.239616 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3326ab1787714e708abfea6cf0509109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.254501 16584 tablet_service.cc:1511] Processing CreateTablet for tablet e0b4dc6fa23d44edb449d1be0842b184 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:19.259266 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee99829f455541f09dac21015b00282e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.259569 16584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0b4dc6fa23d44edb449d1be0842b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.264794 16785 tablet_bootstrap.cc:492] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:19.270797 16658 tablet_service.cc:1511] Processing CreateTablet for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:17:19.259626 16729 tablet_service.cc:1511] Processing CreateTablet for tablet e0b4dc6fa23d44edb449d1be0842b184 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:19.272514 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.272516 16729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0b4dc6fa23d44edb449d1be0842b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.288223 16658 tablet_service.cc:1511] Processing CreateTablet for tablet ee99829f455541f09dac21015b00282e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:17:19.289616 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee99829f455541f09dac21015b00282e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.295522 16731 tablet_service.cc:1511] Processing CreateTablet for tablet 92d5dbc15e7c4691b7ef572175476c3d (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:19.296834 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d5dbc15e7c4691b7ef572175476c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.303979 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3326ab1787714e708abfea6cf0509109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.304200 16577 tablet_service.cc:1511] Processing CreateTablet for tablet 92d5dbc15e7c4691b7ef572175476c3d (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:19.304464 16654 tablet_service.cc:1511] Processing CreateTablet for tablet 3326ab1787714e708abfea6cf0509109 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:17:19.305509 16577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d5dbc15e7c4691b7ef572175476c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.305878 16654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3326ab1787714e708abfea6cf0509109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.310789 16786 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:19.313907 16578 tablet_service.cc:1511] Processing CreateTablet for tablet 446649301a1c45808dccfebfadfed54c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:19.315310 16785 tablet_bootstrap.cc:654] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.315634 16578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446649301a1c45808dccfebfadfed54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.302829 16657 tablet_service.cc:1511] Processing CreateTablet for tablet 8b778f817264440987b35333e6540c72 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:17:19.324352 16657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b778f817264440987b35333e6540c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.337783 16658 tablet_service.cc:1511] Processing CreateTablet for tablet e0b4dc6fa23d44edb449d1be0842b184 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:17:19.339743 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0b4dc6fa23d44edb449d1be0842b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.341676 16661 tablet_service.cc:1511] Processing CreateTablet for tablet 92d5dbc15e7c4691b7ef572175476c3d (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:17:19.344642 16735 tablet_service.cc:1511] Processing CreateTablet for tablet 446649301a1c45808dccfebfadfed54c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:19.344949 16727 tablet_service.cc:1511] Processing CreateTablet for tablet ff94f30d9bba44ee9059993ecbbbf398 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:19.346098 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446649301a1c45808dccfebfadfed54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.346653 16579 tablet_service.cc:1511] Processing CreateTablet for tablet 3386b3f36f454834aaf359817c1adadb (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:19.348150 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff94f30d9bba44ee9059993ecbbbf398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.348800 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3386b3f36f454834aaf359817c1adadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.351953 16584 tablet_service.cc:1511] Processing CreateTablet for tablet ff94f30d9bba44ee9059993ecbbbf398 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:19.353324 16584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff94f30d9bba44ee9059993ecbbbf398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.356040 16786 tablet_bootstrap.cc:654] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.357769 16660 tablet_service.cc:1511] Processing CreateTablet for tablet 446649301a1c45808dccfebfadfed54c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:17:19.368316 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d5dbc15e7c4691b7ef572175476c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.390645 16728 tablet_service.cc:1511] Processing CreateTablet for tablet 3386b3f36f454834aaf359817c1adadb (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:19.392138 16728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3386b3f36f454834aaf359817c1adadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.405844 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446649301a1c45808dccfebfadfed54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.405933 16661 tablet_service.cc:1511] Processing CreateTablet for tablet 3386b3f36f454834aaf359817c1adadb (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:17:19.407397 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3386b3f36f454834aaf359817c1adadb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.363966 16658 tablet_service.cc:1511] Processing CreateTablet for tablet ff94f30d9bba44ee9059993ecbbbf398 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:17:19.419358 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff94f30d9bba44ee9059993ecbbbf398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.420722 16732 tablet_service.cc:1511] Processing CreateTablet for tablet 51e96ec0851c4732b9a3e1272e9fd80f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:19.429543 16785 tablet_bootstrap.cc:492] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:19.430105 16785 ts_tablet_manager.cc:1403] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.166s	user 0.008s	sys 0.009s
I20260709 13:17:19.432991 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e96ec0851c4732b9a3e1272e9fd80f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:19.435473 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:19.441875 16581 tablet_service.cc:1511] Processing CreateTablet for tablet 9f202d4fdd58454a9c8e006ea35dbc46 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:19.443300 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f202d4fdd58454a9c8e006ea35dbc46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.443749 16653 tablet_service.cc:1511] Processing CreateTablet for tablet 51e96ec0851c4732b9a3e1272e9fd80f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:19.445058 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e96ec0851c4732b9a3e1272e9fd80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.451750 16785 raft_consensus.cc:359] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.451987 16585 tablet_service.cc:1511] Processing CreateTablet for tablet 51e96ec0851c4732b9a3e1272e9fd80f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:17:19.452600 16785 raft_consensus.cc:385] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.452931 16785 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.453589 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e96ec0851c4732b9a3e1272e9fd80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.453682 16785 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.462127 16655 tablet_service.cc:1511] Processing CreateTablet for tablet 9f202d4fdd58454a9c8e006ea35dbc46 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:19.463538 16655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f202d4fdd58454a9c8e006ea35dbc46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.467106 16787 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:19.468888 16727 tablet_service.cc:1511] Processing CreateTablet for tablet 9f202d4fdd58454a9c8e006ea35dbc46 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:17:19.470278 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f202d4fdd58454a9c8e006ea35dbc46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.475859 16787 tablet_bootstrap.cc:654] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.489670 16656 tablet_service.cc:1511] Processing CreateTablet for tablet f836a80f84054393b6251b24eaf890f1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:19.493127 16656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f836a80f84054393b6251b24eaf890f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.495304 16786 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:19.495832 16786 ts_tablet_manager.cc:1403] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.185s	user 0.009s	sys 0.028s
I20260709 13:17:19.500582 16787 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:19.511353 16787 ts_tablet_manager.cc:1403] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.045s	user 0.005s	sys 0.008s
I20260709 13:17:19.515849 16621 heartbeater.cc:499] Master 127.15.29.62:37247 was elected leader, sending a full tablet report...
I20260709 13:17:19.479745 16731 tablet_service.cc:1511] Processing CreateTablet for tablet f836a80f84054393b6251b24eaf890f1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:19.512645 16728 tablet_service.cc:1511] Processing CreateTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:19.517417 16728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.518633 16579 tablet_service.cc:1511] Processing CreateTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:19.523553 16785 ts_tablet_manager.cc:1434] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.093s	user 0.016s	sys 0.036s
I20260709 13:17:19.514197 16581 tablet_service.cc:1511] Processing CreateTablet for tablet f836a80f84054393b6251b24eaf890f1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:17:19.502470 16786 raft_consensus.cc:359] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.524464 16786 raft_consensus.cc:385] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.524808 16786 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.515453 16653 tablet_service.cc:1511] Processing CreateTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:17:19.526760 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.528784 16790 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.529286 16790 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.523700 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f836a80f84054393b6251b24eaf890f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.529719 16729 tablet_service.cc:1511] Processing CreateTablet for tablet 73089175ee9c465197a1a11e304346be (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:19.541486 16729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73089175ee9c465197a1a11e304346be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.541755 16787 raft_consensus.cc:359] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.529699 16585 tablet_service.cc:1511] Processing CreateTablet for tablet 73089175ee9c465197a1a11e304346be (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:19.533056 16785 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:19.533509 16584 tablet_service.cc:1511] Processing CreateTablet for tablet 5bfdfe1aa047409c9214ba1f43eb434a (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:19.535252 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.535982 16734 tablet_service.cc:1511] Processing CreateTablet for tablet 5bfdfe1aa047409c9214ba1f43eb434a (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:19.547508 16583 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecf0f1df23742a2a06e910e6b7350a0 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:19.547250 16787 raft_consensus.cc:385] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.547245 16658 tablet_service.cc:1511] Processing CreateTablet for tablet bab84fe76da941dc98416ce3f8c6f35b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:19.549058 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bfdfe1aa047409c9214ba1f43eb434a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.535982 16654 tablet_service.cc:1511] Processing CreateTablet for tablet 5bfdfe1aa047409c9214ba1f43eb434a (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:17:19.549641 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab84fe76da941dc98416ce3f8c6f35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.550348 16654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bfdfe1aa047409c9214ba1f43eb434a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.550037 16661 tablet_service.cc:1511] Processing CreateTablet for tablet 56335fb09b2846e184d61194e1d17f74 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:19.551404 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56335fb09b2846e184d61194e1d17f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.552978 16790 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:19.553735 16584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5bfdfe1aa047409c9214ba1f43eb434a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.554459 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73089175ee9c465197a1a11e304346be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.538663 16652 tablet_service.cc:1511] Processing CreateTablet for tablet 73089175ee9c465197a1a11e304346be (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:17:19.531704 16577 tablet_service.cc:1511] Processing CreateTablet for tablet 56335fb09b2846e184d61194e1d17f74 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:19.542678 16726 tablet_service.cc:1511] Processing CreateTablet for tablet bab84fe76da941dc98416ce3f8c6f35b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:19.544988 16578 tablet_service.cc:1511] Processing CreateTablet for tablet bab84fe76da941dc98416ce3f8c6f35b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:17:19.565632 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab84fe76da941dc98416ce3f8c6f35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.548635 16787 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.557579 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f836a80f84054393b6251b24eaf890f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.539427 16786 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.576685 16659 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecf0f1df23742a2a06e910e6b7350a0 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:19.580348 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecf0f1df23742a2a06e910e6b7350a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.583189 16578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab84fe76da941dc98416ce3f8c6f35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.590168 16727 tablet_service.cc:1511] Processing CreateTablet for tablet ada6718f86ea49aa96bb0fdac1acca8e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:19.591746 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada6718f86ea49aa96bb0fdac1acca8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.593194 16657 tablet_service.cc:1511] Processing CreateTablet for tablet ada6718f86ea49aa96bb0fdac1acca8e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:19.611151 16580 tablet_service.cc:1511] Processing CreateTablet for tablet ada6718f86ea49aa96bb0fdac1acca8e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:17:19.611954 16582 tablet_service.cc:1511] Processing CreateTablet for tablet ecb93faa15cf4f7b853b030dfdc6af28 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:19.612358 16586 tablet_service.cc:1511] Processing CreateTablet for tablet b20d43e167554c06a6ae722c1a39c5d3 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:19.610724 16733 tablet_service.cc:1511] Processing CreateTablet for tablet ecb93faa15cf4f7b853b030dfdc6af28 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:19.594911 16732 tablet_service.cc:1511] Processing CreateTablet for tablet 0ecf0f1df23742a2a06e910e6b7350a0 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:17:19.618032 16733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb93faa15cf4f7b853b030dfdc6af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.632259 16652 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73089175ee9c465197a1a11e304346be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.633107 16657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada6718f86ea49aa96bb0fdac1acca8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.643972 16577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56335fb09b2846e184d61194e1d17f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.648289 16660 tablet_service.cc:1511] Processing CreateTablet for tablet ecb93faa15cf4f7b853b030dfdc6af28 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:17:19.656119 16580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada6718f86ea49aa96bb0fdac1acca8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.658739 16735 tablet_service.cc:1511] Processing CreateTablet for tablet 56335fb09b2846e184d61194e1d17f74 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:17:19.642915 16787 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.661464 16586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20d43e167554c06a6ae722c1a39c5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.671175 16730 tablet_service.cc:1511] Processing CreateTablet for tablet a931fb289f824ff28e993f8590e64b69 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:19.675863 16655 tablet_service.cc:1511] Processing CreateTablet for tablet b20d43e167554c06a6ae722c1a39c5d3 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:19.655012 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecf0f1df23742a2a06e910e6b7350a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.655340 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecf0f1df23742a2a06e910e6b7350a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.684639 16656 tablet_service.cc:1511] Processing CreateTablet for tablet a931fb289f824ff28e993f8590e64b69 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:19.685511 16730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931fb289f824ff28e993f8590e64b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.685964 16656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931fb289f824ff28e993f8590e64b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.691047 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb93faa15cf4f7b853b030dfdc6af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.692127 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:19.693866 16728 tablet_service.cc:1511] Processing CreateTablet for tablet b20d43e167554c06a6ae722c1a39c5d3 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:17:19.695142 16728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20d43e167554c06a6ae722c1a39c5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.696731 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56335fb09b2846e184d61194e1d17f74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.703474 16655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20d43e167554c06a6ae722c1a39c5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.704011 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:19.705694 16561 leader_election.cc:343] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:19.711138 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb93faa15cf4f7b853b030dfdc6af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.716876 16770 heartbeater.cc:499] Master 127.15.29.62:37247 was elected leader, sending a full tablet report...
I20260709 13:17:19.717319 16786 ts_tablet_manager.cc:1434] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.221s	user 0.016s	sys 0.006s
W20260709 13:17:19.718294 16559 leader_election.cc:343] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:19.718727 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:19.719827 16790 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:19.720873 16785 tablet_bootstrap.cc:654] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.724632 16726 tablet_service.cc:1511] Processing CreateTablet for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:19.725996 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.726436 16696 heartbeater.cc:499] Master 127.15.29.62:37247 was elected leader, sending a full tablet report...
I20260709 13:17:19.742595 16653 tablet_service.cc:1511] Processing CreateTablet for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:19.747895 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.747985 16658 tablet_service.cc:1511] Processing CreateTablet for tablet 98936dee7edb4aa5873923a47699a89c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:19.749444 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98936dee7edb4aa5873923a47699a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.783903 16581 tablet_service.cc:1511] Processing CreateTablet for tablet 98936dee7edb4aa5873923a47699a89c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:19.785277 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98936dee7edb4aa5873923a47699a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.787693 16787 ts_tablet_manager.cc:1434] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.272s	user 0.009s	sys 0.006s
I20260709 13:17:19.788679 16787 tablet_bootstrap.cc:492] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:19.792990 16578 tablet_service.cc:1511] Processing CreateTablet for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:17:19.794277 16578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.795910 16787 tablet_bootstrap.cc:654] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.796779 16734 tablet_service.cc:1511] Processing CreateTablet for tablet 98936dee7edb4aa5873923a47699a89c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:17:19.798053 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98936dee7edb4aa5873923a47699a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.799275 16786 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:19.807590 16786 tablet_bootstrap.cc:654] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.807310 16579 tablet_service.cc:1511] Processing CreateTablet for tablet a931fb289f824ff28e993f8590e64b69 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:17:19.808718 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931fb289f824ff28e993f8590e64b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.820200 16585 tablet_service.cc:1511] Processing CreateTablet for tablet f1a71b94ecfb4155a5cfe591077e9a1c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:19.821707 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a71b94ecfb4155a5cfe591077e9a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.837795 16785 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:19.838327 16785 ts_tablet_manager.cc:1403] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.308s	user 0.007s	sys 0.015s
I20260709 13:17:19.834431 16659 tablet_service.cc:1511] Processing CreateTablet for tablet f1a71b94ecfb4155a5cfe591077e9a1c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:19.841343 16785 raft_consensus.cc:359] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.842252 16785 raft_consensus.cc:385] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.842662 16785 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.842931 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a71b94ecfb4155a5cfe591077e9a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.843683 16785 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.846184 16785 ts_tablet_manager.cc:1434] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:19.848428 16785 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:19.857002 16727 tablet_service.cc:1511] Processing CreateTablet for tablet f1a71b94ecfb4155a5cfe591077e9a1c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:17:19.858403 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a71b94ecfb4155a5cfe591077e9a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.862452 16732 tablet_service.cc:1511] Processing CreateTablet for tablet e8b0d179ad714b59b34ab1dc57a57c02 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:19.863891 16660 tablet_service.cc:1511] Processing CreateTablet for tablet e8b0d179ad714b59b34ab1dc57a57c02 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:19.865168 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8b0d179ad714b59b34ab1dc57a57c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.863914 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8b0d179ad714b59b34ab1dc57a57c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.867667 16580 tablet_service.cc:1511] Processing CreateTablet for tablet e8b0d179ad714b59b34ab1dc57a57c02 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:17:19.868955 16580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8b0d179ad714b59b34ab1dc57a57c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.873986 16786 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:19.874564 16786 ts_tablet_manager.cc:1403] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.011s
I20260709 13:17:19.877274 16661 tablet_service.cc:1511] Processing CreateTablet for tablet 173ca11ec288406aa68fe01a80d48e30 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:19.877952 16786 raft_consensus.cc:359] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.878811 16786 raft_consensus.cc:385] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.879743 16786 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.881266 16786 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.883795 16786 ts_tablet_manager.cc:1434] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 13:17:19.884351 16787 tablet_bootstrap.cc:492] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:19.884804 16786 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:19.884848 16787 ts_tablet_manager.cc:1403] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.096s	user 0.020s	sys 0.011s
I20260709 13:17:19.888185 16787 raft_consensus.cc:359] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.888958 16787 raft_consensus.cc:385] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.889348 16787 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.890260 16787 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.891486 16786 tablet_bootstrap.cc:654] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.900004 16733 tablet_service.cc:1511] Processing CreateTablet for tablet 173ca11ec288406aa68fe01a80d48e30 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:19.901557 16733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173ca11ec288406aa68fe01a80d48e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.902717 16585 tablet_service.cc:1511] Processing CreateTablet for tablet 894a0fdf94344f46b0426efbd37a6c40 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:19.904076 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 894a0fdf94344f46b0426efbd37a6c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.878808 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173ca11ec288406aa68fe01a80d48e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.928107 16581 tablet_service.cc:1511] Processing CreateTablet for tablet 173ca11ec288406aa68fe01a80d48e30 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:17:19.929495 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173ca11ec288406aa68fe01a80d48e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:19.948742 16796 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.949276 16796 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.952910 16796 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:19.946516 16785 tablet_bootstrap.cc:654] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:19.969741 16795 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:19.970274 16795 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.972848 16795 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:19.972898 16786 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:19.975021 16786 ts_tablet_manager.cc:1403] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.090s	user 0.009s	sys 0.015s
I20260709 13:17:19.982363 16787 ts_tablet_manager.cc:1434] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.097s	user 0.009s	sys 0.000s
I20260709 13:17:19.990058 16786 raft_consensus.cc:359] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.990797 16786 raft_consensus.cc:385] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:19.991113 16786 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:19.991828 16786 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:19.994294 16786 ts_tablet_manager.cc:1434] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:17:19.995251 16786 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:19.988615 16787 tablet_bootstrap.cc:492] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.016793 16586 tablet_service.cc:1511] Processing CreateTablet for tablet e5e5fb284dac45b2a4c12e2ec38a1784 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:20.018410 16586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5e5fb284dac45b2a4c12e2ec38a1784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.025038 16583 tablet_service.cc:1511] Processing CreateTablet for tablet 374e8578b72347509cc3ea1e6f464f31 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:20.026432 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374e8578b72347509cc3ea1e6f464f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.035885 16582 tablet_service.cc:1511] Processing CreateTablet for tablet 81853b1e217c4b3aa8033dd7e0fdf211 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:20.037302 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81853b1e217c4b3aa8033dd7e0fdf211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.037647 16653 tablet_service.cc:1511] Processing CreateTablet for tablet 894a0fdf94344f46b0426efbd37a6c40 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:20.038952 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 894a0fdf94344f46b0426efbd37a6c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.035885 16658 tablet_service.cc:1511] Processing CreateTablet for tablet 374e8578b72347509cc3ea1e6f464f31 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:20.039143 16661 tablet_service.cc:1511] Processing CreateTablet for tablet e5e5fb284dac45b2a4c12e2ec38a1784 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:20.040328 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374e8578b72347509cc3ea1e6f464f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.053072 16726 tablet_service.cc:1511] Processing CreateTablet for tablet 894a0fdf94344f46b0426efbd37a6c40 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:17:20.059367 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 894a0fdf94344f46b0426efbd37a6c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.060254 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5e5fb284dac45b2a4c12e2ec38a1784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.059894 16659 tablet_service.cc:1511] Processing CreateTablet for tablet 81853b1e217c4b3aa8033dd7e0fdf211 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:20.060629 16786 tablet_bootstrap.cc:654] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.063217 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81853b1e217c4b3aa8033dd7e0fdf211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.069445 16787 tablet_bootstrap.cc:654] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.071559 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:20.073268 16636 leader_election.cc:343] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.074488 16583 tablet_service.cc:1511] Processing CreateTablet for tablet 7c74caaa7fe045f9a0f6ff3b3df97602 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:20.075399 16655 tablet_service.cc:1511] Processing CreateTablet for tablet 7c74caaa7fe045f9a0f6ff3b3df97602 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:20.076004 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c74caaa7fe045f9a0f6ff3b3df97602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.076758 16655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c74caaa7fe045f9a0f6ff3b3df97602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.094424 16728 tablet_service.cc:1511] Processing CreateTablet for tablet e5e5fb284dac45b2a4c12e2ec38a1784 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:17:20.095950 16728 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5e5fb284dac45b2a4c12e2ec38a1784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.101492 16785 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.102089 16785 ts_tablet_manager.cc:1403] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.255s	user 0.016s	sys 0.008s
I20260709 13:17:20.102800 16585 tablet_service.cc:1511] Processing CreateTablet for tablet 0445daac2da44dc5af84baa8514bb195 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:20.105427 16785 raft_consensus.cc:359] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.106114 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0445daac2da44dc5af84baa8514bb195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.106263 16785 raft_consensus.cc:385] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.106775 16785 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.104658 16660 tablet_service.cc:1511] Processing CreateTablet for tablet 0445daac2da44dc5af84baa8514bb195 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:20.108086 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0445daac2da44dc5af84baa8514bb195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.107811 16785 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.109613 16579 tablet_service.cc:1511] Processing CreateTablet for tablet be6c4adb5a2a4a34b6c89e15242b20a1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:20.110513 16732 tablet_service.cc:1511] Processing CreateTablet for tablet 374e8578b72347509cc3ea1e6f464f31 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:17:20.120508 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374e8578b72347509cc3ea1e6f464f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.111219 16735 tablet_service.cc:1511] Processing CreateTablet for tablet 7c74caaa7fe045f9a0f6ff3b3df97602 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:17:20.122599 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c74caaa7fe045f9a0f6ff3b3df97602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.111536 16653 tablet_service.cc:1511] Processing CreateTablet for tablet be6c4adb5a2a4a34b6c89e15242b20a1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:20.112074 16727 tablet_service.cc:1511] Processing CreateTablet for tablet 81853b1e217c4b3aa8033dd7e0fdf211 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:17:20.113915 16734 tablet_service.cc:1511] Processing CreateTablet for tablet be6c4adb5a2a4a34b6c89e15242b20a1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:17:20.130074 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81853b1e217c4b3aa8033dd7e0fdf211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.130898 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6c4adb5a2a4a34b6c89e15242b20a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.130074 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6c4adb5a2a4a34b6c89e15242b20a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.143720 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6c4adb5a2a4a34b6c89e15242b20a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.146469 16785 ts_tablet_manager.cc:1434] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.002s
I20260709 13:17:20.147995 16785 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.168046 16729 tablet_service.cc:1511] Processing CreateTablet for tablet 0445daac2da44dc5af84baa8514bb195 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:17:20.169409 16729 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0445daac2da44dc5af84baa8514bb195. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:20.173913 16478 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62043.2us
I20260709 13:17:20.177472 16787 tablet_bootstrap.cc:492] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.177990 16787 ts_tablet_manager.cc:1403] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.195s	user 0.002s	sys 0.011s
I20260709 13:17:20.179090 16790 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.179576 16790 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.181653 16790 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.186883 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.186950 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.184020 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:20.188241 16561 leader_election.cc:343] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:20.188699 16559 leader_election.cc:343] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.189075 16559 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.189764 16790 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:20.191777 16708 leader_election.cc:343] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.197335 16730 tablet_service.cc:1511] Processing CreateTablet for tablet 6d92a8b6dd5a4c64b749d3505ca2033e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:20.197758 16785 tablet_bootstrap.cc:654] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.198689 16730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d92a8b6dd5a4c64b749d3505ca2033e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.204075 16795 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.204818 16795 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.207651 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.208666 16795 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:17:20.209334 16634 leader_election.cc:343] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.209786 16634 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.210693 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.211762 16731 tablet_service.cc:1511] Processing CreateTablet for tablet f919f8e7484c4f359aedc6ac05b45076 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 13:17:20.212591 16708 leader_election.cc:343] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.213094 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919f8e7484c4f359aedc6ac05b45076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.215051 16796 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.222057 16580 tablet_service.cc:1511] Processing CreateTablet for tablet 6d92a8b6dd5a4c64b749d3505ca2033e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:20.224169 16658 tablet_service.cc:1511] Processing CreateTablet for tablet 6d92a8b6dd5a4c64b749d3505ca2033e (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:17:20.229198 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d92a8b6dd5a4c64b749d3505ca2033e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.231592 16659 tablet_service.cc:1511] Processing CreateTablet for tablet 10731e2d513f466faf0d3f2030af5469 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:20.236493 16580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d92a8b6dd5a4c64b749d3505ca2033e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.238142 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:20.240497 16708 leader_election.cc:343] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.240691 16583 tablet_service.cc:1511] Processing CreateTablet for tablet f919f8e7484c4f359aedc6ac05b45076 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:20.241140 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:20.242002 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919f8e7484c4f359aedc6ac05b45076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.242636 16795 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.243918 16578 tablet_service.cc:1511] Processing CreateTablet for tablet 10731e2d513f466faf0d3f2030af5469 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:20.244143 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.246143 16578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10731e2d513f466faf0d3f2030af5469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.246019 16661 tablet_service.cc:1511] Processing CreateTablet for tablet f919f8e7484c4f359aedc6ac05b45076 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:17:20.247270 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10731e2d513f466faf0d3f2030af5469. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:20.247205 16708 leader_election.cc:343] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.247988 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:20.249089 16795 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.249917 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919f8e7484c4f359aedc6ac05b45076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.250365 16787 raft_consensus.cc:359] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.251713 16787 raft_consensus.cc:385] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.252267 16787 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.254103 16786 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.254624 16786 ts_tablet_manager.cc:1403] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.260s	user 0.014s	sys 0.010s
I20260709 13:17:20.255489 16726 tablet_service.cc:1511] Processing CreateTablet for tablet 10731e2d513f466faf0d3f2030af5469 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:17:20.256865 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10731e2d513f466faf0d3f2030af5469. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.262509 16787 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.270149 16656 tablet_service.cc:1511] Processing CreateTablet for tablet 3551e95f0ec5435f84de7b8f75069b13 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:20.271565 16656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3551e95f0ec5435f84de7b8f75069b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.272410 16787 ts_tablet_manager.cc:1434] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.001s
I20260709 13:17:20.273522 16787 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.280141 16786 raft_consensus.cc:359] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.280905 16786 raft_consensus.cc:385] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.281188 16786 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.281890 16786 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.283715 16581 tablet_service.cc:1511] Processing CreateTablet for tablet 3551e95f0ec5435f84de7b8f75069b13 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:20.284982 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3551e95f0ec5435f84de7b8f75069b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.285332 16786 ts_tablet_manager.cc:1434] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:17:20.286321 16786 tablet_bootstrap.cc:492] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.294325 16787 tablet_bootstrap.cc:654] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.305536 16786 tablet_bootstrap.cc:654] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.309068 16785 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.309551 16785 ts_tablet_manager.cc:1403] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.162s	user 0.013s	sys 0.034s
I20260709 13:17:20.313944 16785 raft_consensus.cc:359] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.315632 16732 tablet_service.cc:1511] Processing CreateTablet for tablet 3551e95f0ec5435f84de7b8f75069b13 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:17:20.316809 16732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3551e95f0ec5435f84de7b8f75069b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.322127 16582 tablet_service.cc:1511] Processing CreateTablet for tablet 92129e39cb38498ea9f0addc0a33f247 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:20.323534 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92129e39cb38498ea9f0addc0a33f247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.326102 16790 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.326592 16790 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.328714 16790 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.330279 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.332381 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.332716 16796 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.333269 16796 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:20.334012 16561 leader_election.cc:343] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:20.334563 16559 leader_election.cc:343] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.335045 16559 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.335853 16658 tablet_service.cc:1511] Processing CreateTablet for tablet 92129e39cb38498ea9f0addc0a33f247 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:20.337684 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92129e39cb38498ea9f0addc0a33f247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.336702 16785 raft_consensus.cc:385] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.339016 16790 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.350651 16796 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.350782 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.350585 16731 tablet_service.cc:1511] Processing CreateTablet for tablet 92129e39cb38498ea9f0addc0a33f247 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:17:20.351477 16596 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:20.351660 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.352335 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92129e39cb38498ea9f0addc0a33f247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.352730 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:17:20.353464 16636 leader_election.cc:343] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.350795 16580 tablet_service.cc:1511] Processing CreateTablet for tablet d104b6d16beb411c9099f23901cebbb2 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:20.354377 16796 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:20.354620 16580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d104b6d16beb411c9099f23901cebbb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.354758 16796 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.355265 16796 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.341892 16785 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.363531 16785 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.365228 16660 tablet_service.cc:1511] Processing CreateTablet for tablet d104b6d16beb411c9099f23901cebbb2 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:20.365868 16785 ts_tablet_manager.cc:1434] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.000s
I20260709 13:17:20.366541 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d104b6d16beb411c9099f23901cebbb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.368273 16785 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.370419 16796 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.379508 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:20.380165 16596 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.394515 16581 tablet_service.cc:1511] Processing CreateTablet for tablet 04d635b36e854e4f9ce8b973de8edafe (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:20.398969 16659 tablet_service.cc:1511] Processing CreateTablet for tablet 04d635b36e854e4f9ce8b973de8edafe (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:20.399444 16726 tablet_service.cc:1511] Processing CreateTablet for tablet d104b6d16beb411c9099f23901cebbb2 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:17:20.400520 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d635b36e854e4f9ce8b973de8edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.400689 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d104b6d16beb411c9099f23901cebbb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.404575 16785 tablet_bootstrap.cc:654] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.415608 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d635b36e854e4f9ce8b973de8edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.416249 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:17:20.417738 16636 leader_election.cc:343] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.418167 16795 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.418671 16795 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.420871 16795 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.421121 16786 tablet_bootstrap.cc:492] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.421679 16796 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.422055 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.422760 16595 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:20.423624 16796 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.424010 16787 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.424189 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:20.424585 16787 ts_tablet_manager.cc:1403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.151s	user 0.015s	sys 0.003s
I20260709 13:17:20.424080 16796 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.425777 16795 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:20.426172 16795 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.426530 16795 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.428136 16796 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.421634 16596 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:20.421650 16786 ts_tablet_manager.cc:1403] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.136s	user 0.028s	sys 0.007s
I20260709 13:17:20.430050 16787 raft_consensus.cc:359] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.430855 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.435462 16660 tablet_service.cc:1511] Processing CreateTablet for tablet b40781eb2d304f9b86f5061ae572e1c7 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:20.437144 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40781eb2d304f9b86f5061ae572e1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.436659 16795 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.438177 16786 raft_consensus.cc:359] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.431854 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:20.438630 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:20.441776 16636 leader_election.cc:343] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:20.443815 16634 leader_election.cc:343] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.444259 16634 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.445508 16813 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.440418 16787 raft_consensus.cc:385] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.451228 16787 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.439216 16786 raft_consensus.cc:385] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.452051 16787 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.452699 16786 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.453020 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.453683 16786 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.435462 16579 tablet_service.cc:1511] Processing CreateTablet for tablet b40781eb2d304f9b86f5061ae572e1c7 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:20.457026 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40781eb2d304f9b86f5061ae572e1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.454272 16787 ts_tablet_manager.cc:1434] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:17:20.461256 16796 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:20.461833 16796 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:20.461474 16726 tablet_service.cc:1511] Processing CreateTablet for tablet 04d635b36e854e4f9ce8b973de8edafe (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:17:20.462212 16735 tablet_service.cc:1511] Processing CreateTablet for tablet b40781eb2d304f9b86f5061ae572e1c7 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:17:20.463233 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d635b36e854e4f9ce8b973de8edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.463924 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40781eb2d304f9b86f5061ae572e1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.462683 16796 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.466341 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:20.469151 16708 leader_election.cc:343] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.477407 16734 tablet_service.cc:1511] Processing CreateTablet for tablet 9fe018a9ac6d466983da014c7b86d36f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:20.478082 16583 tablet_service.cc:1511] Processing CreateTablet for tablet 04867b8a53e645f0b58c55ecec0dac8c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:20.478405 16727 tablet_service.cc:1511] Processing CreateTablet for tablet 04867b8a53e645f0b58c55ecec0dac8c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:20.478828 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fe018a9ac6d466983da014c7b86d36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.479394 16583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04867b8a53e645f0b58c55ecec0dac8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.479638 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04867b8a53e645f0b58c55ecec0dac8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.481431 16785 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.483343 16785 ts_tablet_manager.cc:1403] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.115s	user 0.010s	sys 0.007s
I20260709 13:17:20.482750 16795 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.487980 16580 tablet_service.cc:1511] Processing CreateTablet for tablet 9fe018a9ac6d466983da014c7b86d36f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:20.488758 16787 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.488541 16660 tablet_service.cc:1511] Processing CreateTablet for tablet 04867b8a53e645f0b58c55ecec0dac8c (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:17:20.495811 16659 tablet_service.cc:1511] Processing CreateTablet for tablet 9fe018a9ac6d466983da014c7b86d36f (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:17:20.497426 16785 raft_consensus.cc:359] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.498180 16785 raft_consensus.cc:385] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.498479 16785 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.499370 16727 tablet_service.cc:1511] Processing CreateTablet for tablet 42839b9fe9ae47a5a548348ea9b4eee1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:20.502913 16727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42839b9fe9ae47a5a548348ea9b4eee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.502399 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fe018a9ac6d466983da014c7b86d36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.504877 16656 tablet_service.cc:1511] Processing CreateTablet for tablet 42839b9fe9ae47a5a548348ea9b4eee1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:20.505648 16786 ts_tablet_manager.cc:1434] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.071s	user 0.010s	sys 0.005s
I20260709 13:17:20.511880 16786 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.513197 16578 tablet_service.cc:1511] Processing CreateTablet for tablet abd752e770ac43a799af500f16d446d5 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:20.505722 16731 tablet_service.cc:1511] Processing CreateTablet for tablet e91194e070834298ba7ca9d0bdacc3d7 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:20.518999 16731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91194e070834298ba7ca9d0bdacc3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.519353 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04867b8a53e645f0b58c55ecec0dac8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.507797 16582 tablet_service.cc:1511] Processing CreateTablet for tablet e91194e070834298ba7ca9d0bdacc3d7 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:20.508805 16580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fe018a9ac6d466983da014c7b86d36f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.509001 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:20.520424 16596 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.505757 16661 tablet_service.cc:1511] Processing CreateTablet for tablet e91194e070834298ba7ca9d0bdacc3d7 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:17:20.519084 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:20.523108 16578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752e770ac43a799af500f16d446d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:20.524152 16708 leader_election.cc:343] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.527153 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91194e070834298ba7ca9d0bdacc3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.530927 16596 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:20.534668 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:17:20.535632 16815 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:20.535974 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e91194e070834298ba7ca9d0bdacc3d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.536118 16815 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:20.536998 16815 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.539286 16656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42839b9fe9ae47a5a548348ea9b4eee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.539041 16785 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.542090 16581 tablet_service.cc:1511] Processing CreateTablet for tablet 42839b9fe9ae47a5a548348ea9b4eee1 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:17:20.543267 16581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42839b9fe9ae47a5a548348ea9b4eee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.548003 16726 tablet_service.cc:1511] Processing CreateTablet for tablet abd752e770ac43a799af500f16d446d5 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:20.548542 16658 tablet_service.cc:1511] Processing CreateTablet for tablet abd752e770ac43a799af500f16d446d5 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:17:20.550503 16726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752e770ac43a799af500f16d446d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.551882 16787 tablet_bootstrap.cc:654] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.554611 16785 ts_tablet_manager.cc:1434] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.000s
I20260709 13:17:20.563674 16785 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.564842 16655 tablet_service.cc:1511] Processing CreateTablet for tablet 79e87765622246cc907a538b35c52c64 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:20.566130 16658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752e770ac43a799af500f16d446d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.566663 16655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e87765622246cc907a538b35c52c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.576666 16786 tablet_bootstrap.cc:654] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.591395 16585 tablet_service.cc:1511] Processing CreateTablet for tablet 40930ae2df144d709bfb017b86dfbf2b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:20.591862 16653 tablet_service.cc:1511] Processing CreateTablet for tablet 40930ae2df144d709bfb017b86dfbf2b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:20.593238 16653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40930ae2df144d709bfb017b86dfbf2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.593878 16734 tablet_service.cc:1511] Processing CreateTablet for tablet 40930ae2df144d709bfb017b86dfbf2b (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:17:20.595131 16734 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40930ae2df144d709bfb017b86dfbf2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.595678 16735 tablet_service.cc:1511] Processing CreateTablet for tablet 79e87765622246cc907a538b35c52c64 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:20.597013 16735 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e87765622246cc907a538b35c52c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.603592 16579 tablet_service.cc:1511] Processing CreateTablet for tablet 79e87765622246cc907a538b35c52c64 (DEFAULT_TABLE table=test-workload [id=4a927c90303243679103bb3906337003]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:17:20.613911 16785 tablet_bootstrap.cc:654] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.615097 16790 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.622912 16790 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.627558 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.627390 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.628433 16745 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
W20260709 13:17:20.629546 16559 leader_election.cc:343] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.630160 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:17:20.627386 16790 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.644191 16579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e87765622246cc907a538b35c52c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.646067 16491 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:20.651472 16585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40930ae2df144d709bfb017b86dfbf2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:20.651234 16493 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:20.655979 16815 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.659840 16815 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.656637 16790 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:20.660771 16790 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.661103 16790 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.661789 16815 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.663646 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.664402 16785 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.664867 16785 ts_tablet_manager.cc:1403] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.101s	user 0.012s	sys 0.000s
I20260709 13:17:20.666647 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.667459 16596 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:20.669064 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:20.671141 16815 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.671773 16785 raft_consensus.cc:359] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.672488 16785 raft_consensus.cc:385] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.672798 16785 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.673543 16785 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.678694 16815 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.675729 16790 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.685262 16796 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.685778 16796 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.686105 16790 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.687309 16787 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.689379 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.689558 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:20.687901 16796 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.690186 16745 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.688992 16785 ts_tablet_manager.cc:1434] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:17:20.691397 16785 tablet_bootstrap.cc:492] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
W20260709 13:17:20.694103 16636 leader_election.cc:343] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.694147 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.694804 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:17:20.696094 16634 leader_election.cc:343] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:20.696094 16559 leader_election.cc:343] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.696699 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.697468 16796 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.697585 16786 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.698166 16786 ts_tablet_manager.cc:1403] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.186s	user 0.016s	sys 0.006s
I20260709 13:17:20.703776 16785 tablet_bootstrap.cc:654] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.703881 16745 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:20.689535 16787 ts_tablet_manager.cc:1403] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.214s	user 0.021s	sys 0.010s
I20260709 13:17:20.705263 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:17:20.708243 16822 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:20.709789 16822 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:20.711920 16787 raft_consensus.cc:359] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.712666 16787 raft_consensus.cc:385] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.712954 16787 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.711251 16786 raft_consensus.cc:359] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.714342 16786 raft_consensus.cc:385] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.714638 16786 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.715237 16822 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.715497 16786 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.719180 16787 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.719931 16786 ts_tablet_manager.cc:1434] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 13:17:20.720980 16786 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.725839 16787 ts_tablet_manager.cc:1434] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:17:20.728004 16786 tablet_bootstrap.cc:654] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.729113 16787 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.738000 16785 tablet_bootstrap.cc:492] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.738718 16785 ts_tablet_manager.cc:1403] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.028s
I20260709 13:17:20.741883 16785 raft_consensus.cc:359] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.742319 16787 tablet_bootstrap.cc:654] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.742681 16785 raft_consensus.cc:385] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.743034 16785 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.743099 16493 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:20.744415 16822 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.744850 16822 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.746964 16822 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.749257 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:20.750831 16561 leader_election.cc:343] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.749257 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.743898 16785 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:20.752895 16559 leader_election.cc:343] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.753362 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.755489 16815 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.756065 16786 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.755966 16815 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.756726 16786 ts_tablet_manager.cc:1403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.001s
I20260709 13:17:20.758001 16815 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.762959 16822 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.764847 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.765878 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:20.766312 16708 leader_election.cc:343] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.766638 16785 ts_tablet_manager.cc:1434] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
W20260709 13:17:20.767494 16708 leader_election.cc:343] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.767737 16785 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.768057 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:20.768890 16815 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.765980 16786 raft_consensus.cc:359] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.771502 16786 raft_consensus.cc:385] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.771885 16786 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.772838 16786 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.774314 16785 tablet_bootstrap.cc:654] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.776335 16787 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.776829 16787 ts_tablet_manager.cc:1403] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.019s
I20260709 13:17:20.779784 16786 ts_tablet_manager.cc:1434] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:17:20.779852 16787 raft_consensus.cc:359] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.780680 16786 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.780627 16787 raft_consensus.cc:385] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.781102 16787 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.781911 16787 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.784180 16787 ts_tablet_manager.cc:1434] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:17:20.785454 16787 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.787317 16786 tablet_bootstrap.cc:654] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.791512 16785 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.791785 16787 tablet_bootstrap.cc:654] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.792078 16785 ts_tablet_manager.cc:1403] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.001s
I20260709 13:17:20.795648 16785 raft_consensus.cc:359] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.796485 16785 raft_consensus.cc:385] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.796727 16785 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.797343 16785 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.798370 16786 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.798996 16786 ts_tablet_manager.cc:1403] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.019s	user 0.001s	sys 0.017s
I20260709 13:17:20.801440 16787 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.801927 16787 ts_tablet_manager.cc:1403] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:17:20.802145 16785 ts_tablet_manager.cc:1434] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 13:17:20.801637 16786 raft_consensus.cc:359] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.802561 16786 raft_consensus.cc:385] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.802917 16786 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.803144 16785 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.803781 16786 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.804940 16787 raft_consensus.cc:359] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.805668 16787 raft_consensus.cc:385] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.805866 16796 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.805975 16787 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.806294 16786 ts_tablet_manager.cc:1434] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:17:20.806389 16796 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.807049 16787 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:20.808108 16786 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.808797 16796 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.810501 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.812227 16787 ts_tablet_manager.cc:1434] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:20.812868 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.813308 16787 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
W20260709 13:17:20.818603 16636 leader_election.cc:343] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.818903 16786 tablet_bootstrap.cc:654] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:20.819974 16634 leader_election.cc:343] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.820428 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.821044 16785 tablet_bootstrap.cc:654] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.823339 16787 tablet_bootstrap.cc:654] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.832305 16796 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:20.835067 16787 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.835574 16787 ts_tablet_manager.cc:1403] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 13:17:20.838161 16787 raft_consensus.cc:359] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.838835 16787 raft_consensus.cc:385] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.839141 16787 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.839984 16787 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.842341 16786 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.843115 16786 ts_tablet_manager.cc:1403] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.015s
I20260709 13:17:20.843362 16785 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.843794 16785 ts_tablet_manager.cc:1403] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.004s
I20260709 13:17:20.844166 16787 ts_tablet_manager.cc:1434] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:17:20.845091 16787 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.846135 16785 raft_consensus.cc:359] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.846084 16786 raft_consensus.cc:359] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.846897 16785 raft_consensus.cc:385] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.847016 16786 raft_consensus.cc:385] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.847226 16785 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.847401 16786 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.848055 16785 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.848248 16786 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.850454 16786 ts_tablet_manager.cc:1434] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 13:17:20.851063 16787 tablet_bootstrap.cc:654] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.851519 16786 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.858795 16786 tablet_bootstrap.cc:654] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.864615 16785 ts_tablet_manager.cc:1434] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:17:20.865059 16786 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.865586 16785 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.865674 16786 ts_tablet_manager.cc:1403] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:17:20.869421 16786 raft_consensus.cc:359] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.870411 16786 raft_consensus.cc:385] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.870759 16786 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.871666 16786 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.871706 16785 tablet_bootstrap.cc:654] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.873620 16786 ts_tablet_manager.cc:1434] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:20.875020 16786 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.877221 16787 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.877718 16787 ts_tablet_manager.cc:1403] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.019s
I20260709 13:17:20.880614 16786 tablet_bootstrap.cc:654] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.880369 16787 raft_consensus.cc:359] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.881042 16787 raft_consensus.cc:385] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.881319 16787 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.882161 16787 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.886513 16787 ts_tablet_manager.cc:1434] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:20.887455 16787 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.893069 16787 tablet_bootstrap.cc:654] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.897305 16785 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.897769 16785 ts_tablet_manager.cc:1403] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.032s	user 0.022s	sys 0.007s
I20260709 13:17:20.900451 16785 raft_consensus.cc:359] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.901122 16785 raft_consensus.cc:385] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.901396 16785 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.902146 16785 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.903913 16785 ts_tablet_manager.cc:1434] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 13:17:20.906165 16785 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.906893 16786 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.906927 16787 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.907410 16786 ts_tablet_manager.cc:1403] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.000s
I20260709 13:17:20.907507 16787 ts_tablet_manager.cc:1403] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.008s
I20260709 13:17:20.910188 16786 raft_consensus.cc:359] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.910413 16787 raft_consensus.cc:359] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.910885 16786 raft_consensus.cc:385] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.911165 16787 raft_consensus.cc:385] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.911209 16786 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.911576 16787 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.911851 16785 tablet_bootstrap.cc:654] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.912266 16787 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.912266 16786 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.914463 16787 ts_tablet_manager.cc:1434] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:20.915834 16787 tablet_bootstrap.cc:492] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.922817 16786 ts_tablet_manager.cc:1434] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:17:20.923740 16785 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.923840 16786 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.924269 16785 ts_tablet_manager.cc:1403] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 13:17:20.927011 16785 raft_consensus.cc:359] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.927723 16785 raft_consensus.cc:385] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.928015 16785 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.928143 16787 tablet_bootstrap.cc:654] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.928843 16785 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.931326 16785 ts_tablet_manager.cc:1434] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:20.932366 16785 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.935724 16815 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.936769 16786 tablet_bootstrap.cc:654] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.936441 16815 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.938143 16785 tablet_bootstrap.cc:654] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.940523 16815 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.940416 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:20.941603 16596 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:20.942286 16787 tablet_bootstrap.cc:492] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:20.942206 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.942900 16787 ts_tablet_manager.cc:1403] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.007s
I20260709 13:17:20.942817 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:20.943809 16795 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:20.944136 16795 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.944454 16795 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.945768 16787 raft_consensus.cc:359] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.946404 16787 raft_consensus.cc:385] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.946673 16787 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.947479 16787 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:20.948565 16708 leader_election.cc:343] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.949359 16786 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:20.950706 16786 ts_tablet_manager.cc:1403] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.007s
I20260709 13:17:20.952260 16785 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:20.952376 16822 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:20.952759 16785 ts_tablet_manager.cc:1403] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.012s
I20260709 13:17:20.952834 16822 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.953575 16786 raft_consensus.cc:359] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.954231 16786 raft_consensus.cc:385] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.954492 16786 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.949753 16787 ts_tablet_manager.cc:1434] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260709 13:17:20.955019 16822 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:20.955639 16785 raft_consensus.cc:359] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.955413 16786 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.956452 16785 raft_consensus.cc:385] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:20.956292 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:20.956789 16785 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:20.957403 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:20.957489 16785 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.957926 16786 ts_tablet_manager.cc:1434] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
W20260709 13:17:20.957856 16561 leader_election.cc:343] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.958087 16671 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:20.958904 16786 tablet_bootstrap.cc:492] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:20.959399 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:20.959827 16785 ts_tablet_manager.cc:1434] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:17:20.960480 16822 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:20.960734 16785 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:20.960851 16822 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:20.961242 16822 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.964838 16787 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:20.967825 16785 tablet_bootstrap.cc:654] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.968938 16786 tablet_bootstrap.cc:654] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.972591 16795 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:20.980094 16795 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:20.977501 16787 tablet_bootstrap.cc:654] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:20.980317 16813 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:20.981490 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:20.982002 16796 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:20.983090 16708 leader_election.cc:343] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:20.984511 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:20.985214 16595 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:20.997924 16595 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:21.006434 16796 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.007380 16671 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.007056 16796 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.010317 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.011073 16594 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:21.011931 16815 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.012296 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.012441 16815 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.013168 16813 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.013540 16813 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.013875 16813 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.014554 16815 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.015064 16795 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:21.017556 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.019415 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.023487 16670 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:21.024773 16636 leader_election.cc:343] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.020216 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.026003 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.027611 16795 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.027972 16795 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.028501 16795 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.022850 16796 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.031219 16813 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:21.031260 16708 leader_election.cc:343] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.048807 16815 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:21.049753 16813 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.050933 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:21.053865 16593 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.051360 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:17:21.060631 16636 leader_election.cc:343] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.052143 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:17:21.066732 16593 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:21.041244 16822 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.068526 16815 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.089665 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.090472 16670 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.104460 16790 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.105195 16790 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.107583 16790 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.110033 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.110832 16745 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.112370 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.113391 16790 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.113837 16790 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.114223 16790 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.117072 16854 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.117651 16854 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.123544 16860 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.107497 16670 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.070230 16795 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.116127 16815 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:17:21.127195 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59502.7us
W20260709 13:17:21.119356 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44946.9us
I20260709 13:17:21.138350 16795 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.140208 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:21.141753 16708 leader_election.cc:343] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.124082 16860 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.154168 16860 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.155390 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.156069 16594 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:21.157191 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:21.159197 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.169557 16815 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.168895 16813 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.177196 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.177637 16813 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:21.176813 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:17:21.179764 16561 leader_election.cc:343] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.180202 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.181181 16669 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:21.182687 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.178483 16813 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.183637 16668 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.180297 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.184937 16670 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.187903 16822 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.188753 16822 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.189332 16822 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:21.191236 16860 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.191642 16860 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.191946 16822 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.221807 16786 tablet_bootstrap.cc:492] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:21.191977 16860 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:21.224418 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48253.3us
I20260709 13:17:21.225502 16786 ts_tablet_manager.cc:1403] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.264s	user 0.021s	sys 0.052s
I20260709 13:17:21.217886 16795 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.233259 16795 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:21.240361 16795 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.247289 16795 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.304072 16865 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.327616 16786 raft_consensus.cc:359] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.328236 16786 raft_consensus.cc:385] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.328478 16786 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.329069 16786 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.333360 16786 ts_tablet_manager.cc:1434] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.006s
I20260709 13:17:21.335614 16786 tablet_bootstrap.cc:492] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:21.318548 16795 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.332775 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.339766 16596 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:21.330027 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 95019.9us
I20260709 13:17:21.334010 16813 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.335721 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.333755 16670 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:21.329661 16865 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.355362 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.355589 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.356209 16596 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:21.356212 16744 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:21.357378 16865 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.357914 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.358330 16785 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:21.360073 16785 ts_tablet_manager.cc:1403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.400s	user 0.116s	sys 0.144s
I20260709 13:17:21.353315 16854 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.361109 16859 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.361711 16859 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.362181 16859 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.336498 16790 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.367926 16787 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:21.368420 16787 ts_tablet_manager.cc:1403] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.404s	user 0.043s	sys 0.131s
I20260709 13:17:21.344718 16856 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:17:21.348529 16708 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.348688 16813 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:21.377720 16866 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.379402 16671 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:21.379655 16866 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.380681 16795 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.381592 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.382256 16596 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:21.374023 16856 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.384485 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.385079 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:21.385399 16854 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.385752 16854 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.386073 16854 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.386376 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.386751 16856 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.422379 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.468931 16790 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.500996 16786 tablet_bootstrap.cc:654] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.469869 16822 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:17:21.387075 16866 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.387034 16813 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:21.445785 16871 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.502782 16871 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.503268 16871 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.506778 16787 raft_consensus.cc:359] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:21.507429 16868 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:17:21.507632 16787 raft_consensus.cc:385] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.507952 16787 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
W20260709 13:17:21.386935 16559 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.508644 16787 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:21.509227 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 136078us
I20260709 13:17:21.510634 16787 ts_tablet_manager.cc:1434] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.142s	user 0.006s	sys 0.002s
I20260709 13:17:21.496150 16785 raft_consensus.cc:359] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.511034 16785 raft_consensus.cc:385] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.510722 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.511329 16785 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.511693 16671 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.512019 16785 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.513499 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.514075 16785 ts_tablet_manager.cc:1434] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.154s	user 0.005s	sys 0.000s
I20260709 13:17:21.514209 16671 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.515043 16785 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:21.515753 16559 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.516551 16822 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.516938 16822 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.452315 16860 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.517341 16822 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.518813 16860 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.518734 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.519569 16787 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:21.519569 16671 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.520123 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.520463 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.520725 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.520751 16745 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.521456 16743 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.521486 16866 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.521692 16859 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.522167 16866 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.522797 16866 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.524813 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:21.525408 16596 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.527063 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.528604 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.530974 16859 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.531481 16854 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
W20260709 13:17:21.532215 16708 leader_election.cc:343] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:21.447219 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:21.533718 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:21.534325 16708 leader_election.cc:343] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.378829 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.539043 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:21.535979 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.570454 16741 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.551355 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:21.570588 16813 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:21.571187 16593 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.534689 16854 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:21.571345 16785 tablet_bootstrap.cc:654] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.535274 16671 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.543416 16493 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.537892 16822 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.544337 16744 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.544843 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:21.576411 16594 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.576340 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.576498 16743 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.577071 16670 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.577997 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.534927 16859 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.578962 16790 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.579464 16790 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:21.579260 16859 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.580021 16866 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.580307 16790 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.581331 16859 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.542145 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.582211 16668 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.583706 16670 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.584534 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.584851 16559 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.534739 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:21.418231 16858 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:21.604346 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70033.5us
I20260709 13:17:21.544307 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.606923 16865 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.603794 16860 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.611647 16865 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:21.612551 16865 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:21.607764 16636 leader_election.cc:343] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.610215 16668 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:21.621861 16866 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.623078 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.624821 16743 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:21.605779 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64760.4us
I20260709 13:17:21.631278 16786 tablet_bootstrap.cc:492] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:21.631752 16786 ts_tablet_manager.cc:1403] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.297s	user 0.023s	sys 0.014s
I20260709 13:17:21.623471 16858 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:21.622691 16594 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:21.633251 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.636091 16860 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.636206 16596 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:21.636706 16860 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:21.637588 16860 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.638204 16871 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:21.639340 16742 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 1: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:17:21.639281 16858 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.640715 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.641274 16668 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.641155 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.645442 16822 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.648377 16854 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.648725 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.648910 16854 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:21.649793 16854 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:21.650852 16634 leader_election.cc:343] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.651302 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:21.655216 16787 tablet_bootstrap.cc:654] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.653264 16813 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.658001 16741 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:21.658484 16786 raft_consensus.cc:359] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.659119 16786 raft_consensus.cc:385] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.659387 16786 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.660110 16786 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.662446 16786 ts_tablet_manager.cc:1434] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.000s
I20260709 13:17:21.652333 16593 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:21.665347 16786 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:21.665885 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.666980 16742 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.683038 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:21.683890 16594 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:21.685374 16742 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.686239 16668 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.687386 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.685899 16743 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.688237 16790 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.691792 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:21.693313 16813 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.693919 16813 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:21.695236 16859 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:17:21.701995 16813 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } }
I20260709 13:17:21.694482 16790 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:21.711439 16785 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:21.711944 16785 ts_tablet_manager.cc:1403] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.197s	user 0.024s	sys 0.031s
W20260709 13:17:21.716153 16636 consensus_peers.cc:597] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:21.716981 16785 raft_consensus.cc:359] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.717746 16785 raft_consensus.cc:385] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.718365 16871 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.718492 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.718617 16790 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.718003 16785 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.720364 16708 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:21.721688 16785 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.725839 16785 ts_tablet_manager.cc:1434] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.004s
I20260709 13:17:21.726321 16860 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.726824 16860 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.729604 16785 tablet_bootstrap.cc:492] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:21.723562 16871 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.731449 16860 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.745415 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:21.747573 16708 leader_election.cc:343] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.755095 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:21.755568 16708 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:21.756713 16708 leader_election.cc:343] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.757171 16787 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:21.757232 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:21.757768 16787 ts_tablet_manager.cc:1403] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.238s	user 0.014s	sys 0.027s
I20260709 13:17:21.758147 16745 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.759876 16860 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:21.760713 16871 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.765779 16787 raft_consensus.cc:359] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.766583 16787 raft_consensus.cc:385] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.766937 16787 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.772300 16865 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:21.776468 16787 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.779150 16786 tablet_bootstrap.cc:654] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.779253 16787 ts_tablet_manager.cc:1434] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.006s
I20260709 13:17:21.780756 16787 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:21.781286 16865 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:21.795017 16787 tablet_bootstrap.cc:654] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.795923 16785 tablet_bootstrap.cc:654] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.806326 16492 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.816303 16786 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:21.816907 16786 ts_tablet_manager.cc:1403] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.152s	user 0.001s	sys 0.016s
I20260709 13:17:21.819742 16786 raft_consensus.cc:359] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.820580 16786 raft_consensus.cc:385] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.820920 16786 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.826371 16785 tablet_bootstrap.cc:492] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:21.826836 16785 ts_tablet_manager.cc:1403] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.007s
I20260709 13:17:21.829272 16785 raft_consensus.cc:359] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.829850 16785 raft_consensus.cc:385] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.830073 16785 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.830688 16785 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.833689 16787 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:21.834678 16787 ts_tablet_manager.cc:1403] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.001s
I20260709 13:17:21.837006 16787 raft_consensus.cc:359] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.832895 16786 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.837654 16787 raft_consensus.cc:385] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.837975 16787 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.838281 16491 catalog_manager.cc:5697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.839293 16786 ts_tablet_manager.cc:1434] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:17:21.838671 16787 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.840216 16785 ts_tablet_manager.cc:1434] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.001s
I20260709 13:17:21.840297 16786 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:21.841071 16787 ts_tablet_manager.cc:1434] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:21.841171 16785 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:21.842077 16787 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:21.847683 16786 tablet_bootstrap.cc:654] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.848469 16785 tablet_bootstrap.cc:654] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.848526 16787 tablet_bootstrap.cc:654] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.853128 16858 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.853801 16858 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.856540 16858 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.859298 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.861047 16785 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:21.861537 16785 ts_tablet_manager.cc:1403] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.005s
I20260709 13:17:21.862699 16492 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.864910 16785 raft_consensus.cc:359] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.865595 16785 raft_consensus.cc:385] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.866005 16785 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.866677 16785 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.864928 16492 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.872282 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.868742 16785 ts_tablet_manager.cc:1434] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:17:21.880713 16785 tablet_bootstrap.cc:492] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:21.880613 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:21.881814 16871 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.885528 16785 tablet_bootstrap.cc:654] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.893426 16865 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.894021 16865 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.896329 16865 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.897985 16785 tablet_bootstrap.cc:492] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:21.898456 16785 ts_tablet_manager.cc:1403] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.005s
I20260709 13:17:21.900034 16786 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:21.899983 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.900662 16786 ts_tablet_manager.cc:1403] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.061s	user 0.033s	sys 0.021s
I20260709 13:17:21.900899 16668 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:17:21.902185 16785 raft_consensus.cc:359] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.902842 16785 raft_consensus.cc:385] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.902779 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.903246 16785 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.903927 16785 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.900835 16493 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.905055 16559 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:21.905898 16785 ts_tablet_manager.cc:1434] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:17:21.906599 16865 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.906756 16785 tablet_bootstrap.cc:492] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:21.907896 16813 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.908385 16813 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.910604 16787 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:21.911065 16787 ts_tablet_manager.cc:1403] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.035s
I20260709 13:17:21.910522 16813 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.912305 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.912369 16493 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:21.913668 16787 raft_consensus.cc:359] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.914388 16787 raft_consensus.cc:385] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.914625 16787 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.915526 16787 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.917852 16787 ts_tablet_manager.cc:1434] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:21.917560 16786 raft_consensus.cc:359] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.915539 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.918375 16786 raft_consensus.cc:385] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.918792 16786 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.918838 16787 tablet_bootstrap.cc:492] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:21.919641 16786 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.919849 16636 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:21.921205 16813 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.921819 16786 ts_tablet_manager.cc:1434] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:17:21.922930 16786 tablet_bootstrap.cc:492] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:21.918928 16785 tablet_bootstrap.cc:654] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.926535 16787 tablet_bootstrap.cc:654] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.929178 16786 tablet_bootstrap.cc:654] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.936712 16787 tablet_bootstrap.cc:492] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:21.937404 16787 ts_tablet_manager.cc:1403] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.006s
I20260709 13:17:21.939718 16865 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:21.940604 16790 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.941074 16790 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.941215 16787 raft_consensus.cc:359] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.942231 16787 raft_consensus.cc:385] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.942624 16787 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.943234 16790 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:21.943554 16787 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.944965 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:21.946144 16745 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.945212 16785 tablet_bootstrap.cc:492] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:21.946451 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.947003 16785 ts_tablet_manager.cc:1403] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.008s
I20260709 13:17:21.947098 16668 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:21.947266 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:21.947530 16787 ts_tablet_manager.cc:1434] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:17:21.948570 16787 tablet_bootstrap.cc:492] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:21.949065 16786 tablet_bootstrap.cc:492] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:21.949016 16790 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.949740 16786 ts_tablet_manager.cc:1403] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.004s
I20260709 13:17:21.949689 16790 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.950268 16790 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.950251 16785 raft_consensus.cc:359] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.951043 16785 raft_consensus.cc:385] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.951335 16785 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.952780 16786 raft_consensus.cc:359] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.953436 16786 raft_consensus.cc:385] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.953748 16786 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.954102 16787 tablet_bootstrap.cc:654] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.953938 16491 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.954799 16786 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.955760 16871 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.956199 16871 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.955925 16491 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.957123 16790 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.958917 16785 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.960086 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:21.960610 16668 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.961465 16785 ts_tablet_manager.cc:1434] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:17:21.961509 16871 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:21.962495 16785 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:21.964335 16786 ts_tablet_manager.cc:1434] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:21.965337 16786 tablet_bootstrap.cc:492] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:21.968576 16785 tablet_bootstrap.cc:654] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.969563 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:21.970218 16671 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:21.970571 16787 tablet_bootstrap.cc:492] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:21.968333 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:21.971051 16787 ts_tablet_manager.cc:1403] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.023s	user 0.007s	sys 0.009s
I20260709 13:17:21.957821 16491 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:21.971134 16786 tablet_bootstrap.cc:654] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:21.972150 16708 leader_election.cc:343] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:21.973510 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:21.973881 16787 raft_consensus.cc:359] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.974591 16871 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:21.974726 16787 raft_consensus.cc:385] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.975039 16871 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:21.975201 16787 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.975446 16871 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.976385 16787 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.979525 16787 ts_tablet_manager.cc:1434] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:21.980574 16787 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:21.982447 16786 tablet_bootstrap.cc:492] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:21.982961 16786 ts_tablet_manager.cc:1403] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.007s
I20260709 13:17:21.985731 16786 raft_consensus.cc:359] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.986416 16787 tablet_bootstrap.cc:654] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:21.987200 16668 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:21.988646 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:21.989485 16871 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.986456 16786 raft_consensus.cc:385] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:21.990329 16786 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:21.990928 16822 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:21.991045 16786 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.991855 16866 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:21.992332 16866 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:21.992678 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:21.993189 16745 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.993358 16786 ts_tablet_manager.cc:1434] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:17:21.994386 16786 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:21.991451 16822 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:21.996258 16822 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.000296 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:22.001085 16743 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:22.002420 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:22.003332 16865 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.003490 16786 tablet_bootstrap.cc:654] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.003728 16865 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:22.004139 16865 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:21.994405 16866 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.006163 16866 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.006594 16866 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.008599 16866 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.009855 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:22.011795 16559 leader_election.cc:343] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.012199 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:22.012408 16866 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:22.013571 16561 leader_election.cc:343] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.014061 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:22.017004 16863 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:22.026753 16785 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:22.027319 16785 ts_tablet_manager.cc:1403] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.001s
I20260709 13:17:22.033888 16785 raft_consensus.cc:359] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.035296 16785 raft_consensus.cc:385] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.035586 16785 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.038122 16785 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.038743 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:22.040962 16668 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:22.041105 16785 ts_tablet_manager.cc:1434] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:17:22.041589 16745 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:22.044190 16785 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:22.039220 16865 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.047573 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:22.048225 16670 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.055613 16813 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:22.057341 16785 tablet_bootstrap.cc:654] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.072178 16863 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:22.072837 16858 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:22.080570 16863 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:22.082890 16785 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:22.083477 16785 ts_tablet_manager.cc:1403] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.040s	user 0.006s	sys 0.006s
I20260709 13:17:22.091416 16854 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:22.094954 16858 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:22.100684 16785 raft_consensus.cc:359] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.101577 16785 raft_consensus.cc:385] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.101900 16785 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.102656 16785 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.114070 16863 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.115973 16813 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.127038 16813 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.042456 16790 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.124672 16858 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:22.130518 16912 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.131603 16813 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.135330 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:22.135330 16785 ts_tablet_manager.cc:1434] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.002s
I20260709 13:17:22.120293 16670 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:22.137663 16785 tablet_bootstrap.cc:492] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:22.133700 16863 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.164237 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:22.165216 16863 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.166795 16785 tablet_bootstrap.cc:654] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.151901 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:22.181535 16863 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.180495 16871 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:22.181051 16865 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.188198 16863 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:22.198736 16636 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:22.205235 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:22.206091 16669 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.131264 16912 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.208098 16813 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:22.200517 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:22.225476 16559 leader_election.cc:343] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:22.232019 16786 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:22.216266 16916 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.094s
I20260709 13:17:22.232718 16786 ts_tablet_manager.cc:1403] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.239s	user 0.035s	sys 0.051s
I20260709 13:17:22.221518 16863 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.240648 16786 raft_consensus.cc:359] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.238246 16813 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:22.243238 16786 raft_consensus.cc:385] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.252242 16786 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.254773 16787 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:22.255347 16787 ts_tablet_manager.cc:1403] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.275s	user 0.055s	sys 0.136s
I20260709 13:17:22.253209 16786 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.253127 16912 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.258237 16786 ts_tablet_manager.cc:1434] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:17:22.258248 16787 raft_consensus.cc:359] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.259852 16787 raft_consensus.cc:385] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.260251 16787 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.261524 16786 tablet_bootstrap.cc:492] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:22.265211 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:22.268456 16671 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:22.275804 16785 tablet_bootstrap.cc:492] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:22.277621 16785 ts_tablet_manager.cc:1403] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.009s
I20260709 13:17:22.276249 16494 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:22.261695 16787 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.310961 16787 ts_tablet_manager.cc:1434] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.003s
I20260709 13:17:22.333643 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:22.334542 16741 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:22.335644 16918 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.338183 16918 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.345772 16787 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:22.322365 16854 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:17:22.354739 16785 raft_consensus.cc:359] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.359742 16785 raft_consensus.cc:385] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.363096 16785 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.363807 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:22.498196 16859 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.498919 16741 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.498798 16859 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:22.502254 16904 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:17:22.532462 16918 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.536583 16669 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:22.371939 16860 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:22.520995 16670 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.541070 16785 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.524501 16742 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.584424 16786 tablet_bootstrap.cc:654] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.667133 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:22.668184 16594 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.673875 16859 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:22.743016 16785 ts_tablet_manager.cc:1434] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.464s	user 0.006s	sys 0.004s
I20260709 13:17:22.749251 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:17:22.785256 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 118364us
I20260709 13:17:22.553242 16860 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:22.844707 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:22.845659 16740 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:22.667176 16916 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.149s
I20260709 13:17:22.806443 16918 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.894125 16918 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:22.987208 16923 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:17:23.016716 16871 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:23.018182 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:23.027783 16860 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:23.038592 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:23.039865 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:22.323096 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:17:23.074362 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 764151us
I20260709 13:17:23.075920 16916 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:22.991171 16918 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.104087 16787 tablet_bootstrap.cc:654] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.098937 16860 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.113750 16860 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:23.096489 16916 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:23.089535 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 71459.7us
I20260709 13:17:23.120229 16916 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:23.125453 16743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.025s	sys 0.027s
W20260709 13:17:23.135301 16743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.025s	sys 0.028s
I20260709 13:17:23.136790 16742 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
W20260709 13:17:23.130204 16824 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.077s	sys 0.169s
W20260709 13:17:23.138151 16824 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.077s	sys 0.169s
I20260709 13:17:23.156708 16785 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:23.114809 16860 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.182257 16669 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.184084 16916 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.186270 16916 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:23.188381 16561 leader_election.cc:343] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.190899 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:23.191525 16743 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.192088 16911 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.230240 16916 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:23.235020 16741 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.735s	user 0.011s	sys 0.001s
W20260709 13:17:23.305083 16670 consensus_meta.cc:287] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.000s
I20260709 13:17:23.302618 16916 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.318508 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:23.561192 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:23.561810 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 184654us
I20260709 13:17:23.597759 16930 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:17:23.597937 16636 leader_election.cc:343] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.598347 16930 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.600616 16930 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.589533 16595 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:23.605342 16926 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.295s	user 0.000s	sys 0.000s
I20260709 13:17:23.584147 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:23.606571 16666 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:23.580569 16922 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:17:23.606783 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 28) took 1349 ms. Trace:
I20260709 13:17:23.609669 16911 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:17:23.632632 16930 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.631359 16918 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.701380 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:23.702098 16664 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.635910 16911 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.630970 16922 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:23.710937 16925 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
I20260709 13:17:23.712565 16922 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:23.669694 16863 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:17:23.713215 16863 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.717985 16787 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:23.718609 16787 ts_tablet_manager.cc:1403] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 1.407s	user 0.081s	sys 0.108s
I20260709 13:17:23.699204 16932 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:23.671128 16930 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:23.723510 16924 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.344s	user 0.000s	sys 0.001s
I20260709 13:17:23.731803 16916 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.756248 16787 raft_consensus.cc:359] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.737311 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 32) took 1482 ms. Trace:
I20260709 13:17:23.757004 16787 raft_consensus.cc:385] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.727790 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:23.759332 16787 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.753507 16863 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.760649 16787 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:23.771190 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49653.1us
I20260709 13:17:23.775699 16785 tablet_bootstrap.cc:654] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.761307 16863 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:23.780609 16787 ts_tablet_manager.cc:1434] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 13:17:23.781533 16787 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:23.763049 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:23.787804 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66977.6us
I20260709 13:17:23.760902 16946 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:17:23.729794 16932 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.796805 16911 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.629958 16596 rpcz_store.cc:276] 0709 13:17:22.257865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.258946 (+  1081us) service_pool.cc:224] Handling call
0709 13:17:22.264158 (+  5212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.288384 (+ 24226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.288423 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.288944 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.531373 (+242429us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:17:22.531430 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.531526 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.606568 (+1075042us) raft_consensus.cc:1685] finished
0709 13:17:23.606586 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.606723 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23772,"raft.queue_time_us":27937,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":170,"thread_start_us":242007,"threads_started":1,"wal-append.queue_time_us":959090}
W20260709 13:17:23.800149 16708 leader_election.cc:343] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:23.800387 16930 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:23.801978 16743 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:17:23.802223 16664 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:23.803020 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 26) took 1555 ms. Trace:
I20260709 13:17:23.780119 16863 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.805631 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:23.806200 16941 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.806754 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:23.797458 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:23.815757 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 30) took 1446 ms. Trace:
I20260709 13:17:23.822999 16665 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:23.759132 16668 rpcz_store.cc:276] 0709 13:17:22.252820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.258686 (+  5866us) service_pool.cc:224] Handling call
0709 13:17:22.264249 (+  5563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.264643 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.264675 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.265241 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.494801 (+229560us) raft_consensus.cc:1645] Marking committed up to 0
0709 13:17:22.498612 (+  3811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:22.498696 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.727539 (+1228843us) raft_consensus.cc:1685] finished
0709 13:17:23.727559 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.735311 (+  7752us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2606,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":5872,"spinlock_wait_cycles":85760,"thread_start_us":101712,"threads_started":1,"wal-append.queue_time_us":977447}
I20260709 13:17:23.810410 16743 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:23.804958 16595 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:23.823448 16787 tablet_bootstrap.cc:654] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:23.794180 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62713.6us
I20260709 13:17:23.812527 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:23.827935 16941 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.852614 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 26) took 1484 ms. Trace:
I20260709 13:17:23.885654 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:23.889245 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:23.808586 16946 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.880515 16941 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:23.871538 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 63736.7us
I20260709 13:17:23.889446 16951 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:17:23.823053 16670 rpcz_store.cc:276] 0709 13:17:22.496446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.520326 (+ 23880us) service_pool.cc:224] Handling call
0709 13:17:22.520789 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:23.565942 (+1045153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.568976 (+  3034us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:23.608584 (+ 39608us) log.cc:844] Serialized 26 byte log entry
0709 13:17:23.695697 (+ 87113us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:23.715599 (+ 19902us) log.cc:844] Serialized 14 byte log entry
0709 13:17:23.727047 (+ 11448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.729319 (+  2272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.804722 (+ 75403us) raft_consensus.cc:1685] finished
0709 13:17:23.808444 (+  3722us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.813062 (+  4618us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9787,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":4666,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":3544,"thread_start_us":33847,"threads_started":1,"wal-append.queue_time_us":143833}
I20260709 13:17:23.904201 16951 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.906217 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:23.897626 16911 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.813350 16745 rpcz_store.cc:276] 0709 13:17:22.247403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.250130 (+  2727us) service_pool.cc:224] Handling call
0709 13:17:22.250451 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:22.250795 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:22.250825 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:22.251307 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:17:22.747947 (+496640us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:23.282263 (+534316us) log.cc:844] Serialized 14 byte log entry
0709 13:17:23.308221 (+ 25958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.313188 (+  4967us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.802766 (+489578us) raft_consensus.cc:1685] finished
0709 13:17:23.802792 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.802957 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":191,"prepare.queue_time_us":843066,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":20591,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":4352,"thread_start_us":983186,"threads_started":2,"wal-append.queue_time_us":986345}
I20260709 13:17:23.883942 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:23.906700 16946 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:23.914808 16911 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.919682 16949 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:23.880766 16741 rpcz_store.cc:276] 0709 13:17:22.491330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.494848 (+  3518us) service_pool.cc:224] Handling call
0709 13:17:22.498664 (+  3816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:23.539973 (+1041309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:23.561668 (+ 21695us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:23.592177 (+ 30509us) log.cc:844] Serialized 26 byte log entry
0709 13:17:23.735044 (+142867us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:23.756856 (+ 21812us) log.cc:844] Serialized 14 byte log entry
0709 13:17:23.767156 (+ 10300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:23.775948 (+  8792us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:23.816160 (+ 40212us) raft_consensus.cc:1685] finished
0709 13:17:23.824665 (+  8505us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:23.829295 (+  4630us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8751,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":175876,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":32357,"thread_start_us":65475,"threads_started":1,"wal-append.queue_time_us":178099}
I20260709 13:17:23.910560 16871 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:17:23.929580 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:23.929860 16911 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.930207 16871 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.910507 16932 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:23.931818 16948 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:23.927382 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54153.6us
I20260709 13:17:23.932411 16948 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.932928 16948 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.933670 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:23.903298 16736 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:23.914680 16863 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.927269 16947 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.936578 16911 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:23.940567 16935 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:23.942484 16935 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:23.920256 16949 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.941973 16947 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.943192 16947 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.959653 16786 tablet_bootstrap.cc:492] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:23.960439 16786 ts_tablet_manager.cc:1403] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 1.700s	user 0.051s	sys 0.036s
I20260709 13:17:23.957791 16595 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:23.961660 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:23.961953 16935 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.967347 16949 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.967638 16951 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:23.970459 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:23.963225 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 91162.9us
I20260709 13:17:23.959642 16941 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.979079 16785 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:23.979519 16785 ts_tablet_manager.cc:1403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.823s	user 0.023s	sys 0.000s
I20260709 13:17:23.968802 16559 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:23.983263 16949 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:23.983582 16949 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:23.983865 16949 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:23.978600 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:23.994794 16786 raft_consensus.cc:359] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.995630 16786 raft_consensus.cc:385] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.995450 16949 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:23.997941 16949 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:23.996053 16786 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.990676 16958 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:24.007751 16958 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.008232 16958 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.003580 16786 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.011248 16786 ts_tablet_manager.cc:1434] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.051s	user 0.001s	sys 0.007s
I20260709 13:17:24.012094 16786 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.008948 16948 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.029699 16941 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.032612 16948 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.034917 16787 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.035526 16787 ts_tablet_manager.cc:1403] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.254s	user 0.013s	sys 0.007s
I20260709 13:17:24.026327 16947 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.045889 16947 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.049242 16958 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.052899 16958 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.089900 16958 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.090493 16958 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.094592 16958 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.166374 16786 tablet_bootstrap.cc:654] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.166075 16787 raft_consensus.cc:359] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.166785 16787 raft_consensus.cc:385] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.167558 16787 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.171687 16787 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.174201 16787 ts_tablet_manager.cc:1434] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.138s	user 0.009s	sys 0.000s
I20260709 13:17:24.175143 16787 tablet_bootstrap.cc:492] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.176184 16786 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.176687 16786 ts_tablet_manager.cc:1403] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.165s	user 0.015s	sys 0.002s
W20260709 13:17:24.178139 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 195967us
I20260709 13:17:24.179116 16786 raft_consensus.cc:359] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.179638 16786 raft_consensus.cc:385] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.179877 16786 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.180444 16786 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.181969 16787 tablet_bootstrap.cc:654] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.181995 16786 ts_tablet_manager.cc:1434] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.000s
I20260709 13:17:24.183122 16786 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.189509 16786 tablet_bootstrap.cc:654] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.192191 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.192946 16736 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.194799 16787 tablet_bootstrap.cc:492] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.195295 16787 ts_tablet_manager.cc:1403] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 13:17:24.195376 16786 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.195998 16786 ts_tablet_manager.cc:1403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:17:24.198154 16787 raft_consensus.cc:359] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.198850 16787 raft_consensus.cc:385] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.199555 16787 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.200343 16787 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.203219 16787 ts_tablet_manager.cc:1434] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:24.203341 16736 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.204176 16787 tablet_bootstrap.cc:492] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.204810 16871 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.205387 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.199034 16786 raft_consensus.cc:359] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.206302 16786 raft_consensus.cc:385] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.206640 16786 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.207048 16947 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.207588 16947 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:24.207376 16786 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.208492 16947 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.210309 16787 tablet_bootstrap.cc:654] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.210443 16786 ts_tablet_manager.cc:1434] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:17:24.211916 16786 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.218633 16786 tablet_bootstrap.cc:654] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.224296 16786 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.224921 16786 ts_tablet_manager.cc:1403] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:17:24.227006 16787 tablet_bootstrap.cc:492] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.227563 16787 ts_tablet_manager.cc:1403] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.004s
I20260709 13:17:24.229910 16786 raft_consensus.cc:359] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.230204 16787 raft_consensus.cc:359] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.230749 16786 raft_consensus.cc:385] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.231002 16787 raft_consensus.cc:385] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.231246 16786 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.232510 16786 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.231349 16787 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.234045 16787 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.236169 16787 ts_tablet_manager.cc:1434] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:17:24.237150 16787 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.239353 16786 ts_tablet_manager.cc:1434] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:17:24.240422 16786 tablet_bootstrap.cc:492] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.244009 16787 tablet_bootstrap.cc:654] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.253765 16786 tablet_bootstrap.cc:654] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:24.253360 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 270740us
I20260709 13:17:24.259666 16786 tablet_bootstrap.cc:492] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.260607 16786 ts_tablet_manager.cc:1403] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.000s
I20260709 13:17:24.263325 16918 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.263451 16786 raft_consensus.cc:359] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.264175 16786 raft_consensus.cc:385] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.264441 16786 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.264377 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.265216 16786 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.266008 16787 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.266472 16787 ts_tablet_manager.cc:1403] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.009s
I20260709 13:17:24.267352 16786 ts_tablet_manager.cc:1434] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 13:17:24.268268 16786 tablet_bootstrap.cc:492] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.268630 16787 raft_consensus.cc:359] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.269059 16787 raft_consensus.cc:385] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.269210 16787 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.269887 16787 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.271731 16787 ts_tablet_manager.cc:1434] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:24.272624 16787 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.275154 16786 tablet_bootstrap.cc:654] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.277909 16787 tablet_bootstrap.cc:654] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.280632 16786 tablet_bootstrap.cc:492] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.281152 16786 ts_tablet_manager.cc:1403] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:17:24.282593 16787 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.283037 16787 ts_tablet_manager.cc:1403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:17:24.283826 16786 raft_consensus.cc:359] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.284543 16786 raft_consensus.cc:385] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.284804 16786 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.285528 16786 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.286428 16787 raft_consensus.cc:359] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.287186 16787 raft_consensus.cc:385] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.287533 16787 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.287722 16786 ts_tablet_manager.cc:1434] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:17:24.288219 16787 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.290530 16787 ts_tablet_manager.cc:1434] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:24.291520 16787 tablet_bootstrap.cc:492] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.288694 16786 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.297494 16787 tablet_bootstrap.cc:654] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.298645 16786 tablet_bootstrap.cc:654] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.304390 16786 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.305053 16787 tablet_bootstrap.cc:492] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.305297 16786 ts_tablet_manager.cc:1403] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.005s
I20260709 13:17:24.305563 16787 ts_tablet_manager.cc:1403] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:17:24.307981 16786 raft_consensus.cc:359] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.308331 16787 raft_consensus.cc:359] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.308713 16786 raft_consensus.cc:385] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.308928 16787 raft_consensus.cc:385] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.308976 16786 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.309278 16787 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.309680 16786 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.309942 16787 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.311857 16786 ts_tablet_manager.cc:1434] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:17:24.312209 16787 ts_tablet_manager.cc:1434] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:17:24.312867 16786 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.313135 16787 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.319214 16786 tablet_bootstrap.cc:654] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.319219 16787 tablet_bootstrap.cc:654] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.324575 16787 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.325057 16787 ts_tablet_manager.cc:1403] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:17:24.325989 16786 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.326417 16786 ts_tablet_manager.cc:1403] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.014s	user 0.007s	sys 0.004s
I20260709 13:17:24.361802 16932 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.362365 16932 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.364470 16932 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.385262 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:24.396715 16594 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.728s	user 0.013s	sys 0.009s
I20260709 13:17:24.397138 16594 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
W20260709 13:17:24.397748 16594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 28) took 1813 ms (client timeout 2148 ms). Trace:
W20260709 13:17:24.398094 16594 rpcz_store.cc:269] 0709 13:17:22.583882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.635800 (+ 51918us) service_pool.cc:224] Handling call
0709 13:17:24.397685 (+1761885us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:17:24.408386 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.409013 16741 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.413429 16930 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.413740 16939 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.414016 16930 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.414285 16939 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.414810 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.416158 16930 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.416478 16939 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.418593 16787 raft_consensus.cc:359] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.418851 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.419162 16741 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:17:24.419330 16787 raft_consensus.cc:385] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.419523 16745 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:24.419725 16787 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.420405 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.420398 16787 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.421303 16930 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:24.421443 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.422188 16930 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:24.422233 16958 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.422545 16787 ts_tablet_manager.cc:1434] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.097s	user 0.009s	sys 0.000s
I20260709 13:17:24.422763 16958 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.416356 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.423195 16958 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.423553 16787 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.428439 16787 tablet_bootstrap.cc:654] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.430209 16958 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.431571 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.432197 16742 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.433022 16930 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.434094 16958 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.437361 16787 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.437882 16787 ts_tablet_manager.cc:1403] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 13:17:24.440737 16787 raft_consensus.cc:359] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.441411 16787 raft_consensus.cc:385] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.441684 16787 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.442468 16787 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.443544 16786 raft_consensus.cc:359] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.444187 16786 raft_consensus.cc:385] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.444443 16786 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.444867 16787 ts_tablet_manager.cc:1434] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:24.446329 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.449589 16494 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 24) took 2186 ms. Trace:
I20260709 13:17:24.456292 16787 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.445036 16786 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.457235 16859 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:24.457700 16859 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.458020 16859 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.458765 16786 ts_tablet_manager.cc:1434] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.132s	user 0.007s	sys 0.004s
I20260709 13:17:24.445448 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.459636 16785 raft_consensus.cc:359] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.461525 16742 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.456485 16494 rpcz_store.cc:276] 0709 13:17:22.262805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:22.263019 (+   214us) service_pool.cc:224] Handling call
0709 13:17:24.449520 (+2186501us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:22.296412 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:22.326547 (+ 30135us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:22.326593 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:17:22.326613 (+    20us) tablet.cc:662] Decoding operations
0709 13:17:22.327170 (+   557us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:22.327259 (+    89us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:22.327313 (+    54us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:17:22.327557 (+   244us) tablet.cc:701] Row locks acquired
0709 13:17:22.327584 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:17:22.327822 (+   238us) write_op.cc:270] Start()
0709 13:17:22.327977 (+   155us) write_op.cc:276] Timestamp: P: 1783603042327744 usec, L: 0
0709 13:17:22.328025 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 13:17:22.328926 (+   901us) log.cc:844] Serialized 534 byte log entry
0709 13:17:23.393487 (+1064561us) op_driver.cc:464] REPLICATION: finished
0709 13:17:23.753400 (+359913us) write_op.cc:301] APPLY: starting
0709 13:17:23.753508 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:23.756924 (+  3416us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:23.775491 (+ 18567us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:23.785015 (+  9524us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:23.797302 (+ 12287us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:17:23.797364 (+    62us) write_op.cc:312] APPLY: finished
0709 13:17:23.798735 (+  1371us) log.cc:844] Serialized 22 byte log entry
0709 13:17:23.798947 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:23.799040 (+    93us) write_op.cc:454] Released schema lock
0709 13:17:23.799082 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:23.799382 (+   300us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:23.799408 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11135,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":196135,"apply.run_cpu_time_us":2302,"apply.run_wall_time_us":53065,"num_ops":1,"prepare.queue_time_us":145,"prepare.run_cpu_time_us":3720,"prepare.run_wall_time_us":35233,"replication_time_us":983697,"thread_start_us":82164,"threads_started":1}]]}
I20260709 13:17:24.461488 16785 raft_consensus.cc:385] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.465658 16785 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.466240 16785 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.459710 16786 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.467188 16787 tablet_bootstrap.cc:654] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.472843 16787 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
W20260709 13:17:24.473249 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 488109us
I20260709 13:17:24.473869 16787 ts_tablet_manager.cc:1403] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.002s
I20260709 13:17:24.475437 16934 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.476044 16934 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.476807 16787 raft_consensus.cc:359] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.477538 16787 raft_consensus.cc:385] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.477888 16787 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.478019 16934 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.475368 16742 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.478526 16932 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.475200 16859 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.481009 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 30) took 1479 ms. Trace:
I20260709 13:17:24.481650 16786 tablet_bootstrap.cc:654] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.478833 16787 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.481969 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.483669 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.484500 16741 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:17:24.484599 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.485176 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.484391 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.485781 16743 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:24.487210 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.481412 16593 rpcz_store.cc:276] 0709 13:17:23.097726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:23.108424 (+ 10698us) service_pool.cc:224] Handling call
0709 13:17:23.809752 (+701328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:17:23.824828 (+ 15076us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:17:23.905441 (+ 80613us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.480662 (+575221us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:17:24.480715 (+    53us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:24.480760 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.480828 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.480948 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":193497,"prepare.queue_time_us":89373,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":653,"thread_start_us":413226,"threads_started":2,"wal-append.queue_time_us":136064}
I20260709 13:17:24.487910 16742 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.488081 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.489081 16930 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.489393 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.489588 16930 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:24.516438 16859 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.516580 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.517136 16930 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.517925 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.522085 16787 ts_tablet_manager.cc:1434] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 13:17:24.523198 16787 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.524554 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.524935 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.526665 16859 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:17:24.527220 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.527206 16859 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.486636 16932 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.528636 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:24.529417 16663 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.529989 16932 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.532416 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.533285 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.533648 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.538686 16670 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:24.544034 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:24.544966 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.545759 16745 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:17:24.545634 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.547596 16743 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:24.549556 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.550416 16932 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:24.550851 16671 raft_consensus.cc:2410] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:17:24.550806 16932 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:24.551334 16932 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.530782 16859 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.553920 16922 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.559341 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:24.559557 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.560369 16744 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:24.561093 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:24.562914 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.565826 16671 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:17:24.568943 16785 ts_tablet_manager.cc:1434] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.589s	user 0.013s	sys 0.000s
I20260709 13:17:24.569012 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:24.569403 16785 ts_tablet_manager.cc:1467] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet startup took 589ms
I20260709 13:17:24.569851 16588 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 1: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:17:24.570600 16958 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.571074 16958 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.571280 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.573185 16958 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.573645 16958 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.573556 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.574445 16590 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 1: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:17:24.574131 16958 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.575198 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.575280 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.576012 16594 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 1: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:17:24.576113 16595 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 1: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:17:24.576851 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:24.577992 16559 leader_election.cc:343] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.585894 16663 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:24.578073 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.578073 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.588423 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.588541 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:24.588835 16589 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:24.589042 16743 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:24.590293 16860 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.590930 16634 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.592320 16947 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.592790 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.590307 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.593753 16742 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:24.593433 16591 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.594030 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:24.569712 16785 ts_tablet_manager.cc:1469] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trace:
0709 13:17:19.511740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:19.517742 (+  6002us) service_pool.cc:224] Handling call
0709 13:17:19.533910 (+ 16168us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:17:19.534019 (+   109us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:17:19.664261 (+130242us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:19.720725 (+ 56464us) tablet_replica.cc:206] Creating consensus instance
0709 13:17:19.721164 (+   439us) inbound_call.cc:177] Queueing success response
0709 13:17:23.156274 (+3435110us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:17:23.979767 (+823493us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:17:24.006328 (+ 26561us) tablet_replica.cc:261] Starting instrumentation
0709 13:17:24.007010 (+   682us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":548673,"raft.queue_time_us":100927,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"tablet-open.queue_time_us":3435160}
I20260709 13:17:24.596521 16785 tablet_bootstrap.cc:492] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:24.598275 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:24.571313 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.605563 16860 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.608279 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.604460 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.610597 16953 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.611107 16953 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.599852 16744 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.617341 16961 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:17:24.615072 16911 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:24.617887 16961 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.613188 16953 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.618424 16863 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.619683 16559 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.622663 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.623276 16745 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:24.620430 16961 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.621296 16939 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.624733 16939 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.626333 16939 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.621641 16958 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.620136 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.624763 16948 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:24.628554 16932 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.628419 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.629931 16559 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.630483 16932 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.630795 16939 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.631098 16935 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.631327 16939 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.631601 16916 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:24.631943 16916 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:24.631875 16935 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.635708 16958 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.613870 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:24.631882 16939 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.638439 16859 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:17:24.638573 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.639329 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.639286 16859 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.640786 16947 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.637933 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.641676 16932 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.642625 16935 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.645604 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.645479 16922 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.646101 16935 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.646456 16935 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.648594 16591 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.638017 16916 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.651152 16787 tablet_bootstrap.cc:654] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.636649 16958 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.652633 16916 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.653805 16958 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.654702 16708 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.655467 16922 raft_consensus.cc:2764] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:17:24.655920 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.656773 16922 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.657496 16922 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:24.658329 16922 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.659332 16739 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:24.640468 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.660043 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:17:24.661851 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:24.641909 16859 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.663937 16948 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.664242 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.664515 16948 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:24.664891 16663 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:24.665392 16948 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.666042 16911 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.666388 16559 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.667205 16863 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.667642 16559 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.668431 16863 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.670405 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 32) took 1049 ms. Trace:
I20260709 13:17:24.671804 16785 tablet_bootstrap.cc:654] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.671274 16634 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.672544 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:24.673979 16636 leader_election.cc:343] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.674304 16935 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.680567 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.681432 16859 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.681852 16859 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.681941 16947 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.682241 16859 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.670848 16592 rpcz_store.cc:276] 0709 13:17:23.594143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:24.259339 (+665196us) service_pool.cc:224] Handling call
0709 13:17:24.259737 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:24.482887 (+223150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:24.486082 (+  3195us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:24.487170 (+  1088us) log.cc:844] Serialized 26 byte log entry
0709 13:17:24.587518 (+100348us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:17:24.620918 (+ 33400us) log.cc:844] Serialized 14 byte log entry
0709 13:17:24.633450 (+ 12532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:17:24.633522 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:17:24.633562 (+    40us) raft_consensus.cc:1685] finished
0709 13:17:24.633576 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:17:24.633709 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":267097,"prepare.queue_time_us":11801,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":296772,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":170,"thread_start_us":93108,"threads_started":1,"wal-append.queue_time_us":95399}
I20260709 13:17:24.688112 16932 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.688664 16932 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.689448 16786 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:24.689919 16786 ts_tablet_manager.cc:1403] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.230s	user 0.031s	sys 0.012s
I20260709 13:17:24.676116 16930 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.691875 16932 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.692575 16935 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.693075 16859 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.696606 16786 raft_consensus.cc:359] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.697247 16786 raft_consensus.cc:385] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.697501 16786 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.699374 16947 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.699743 16939 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.676074 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.703992 16786 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.710991 16935 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.713032 16708 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:24.717396 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48607.7us
I20260709 13:17:24.718149 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.718941 16935 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.719615 16948 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.720860 16948 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:24.721300 16948 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:24.721426 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.722312 16666 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:17:24.701642 16634 leader_election.cc:343] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.696030 16953 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.731555 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.732903 16939 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.733624 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:24.740060 16786 ts_tablet_manager.cc:1434] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.006s
I20260709 13:17:24.722460 16948 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.741341 16786 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:24.739324 16670 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:24.730129 16744 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:17:24.739213 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.748425 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.749087 16743 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.749581 16948 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:24.755959 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.756111 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:24.748519 16669 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:17:24.756045 16739 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:17:24.743808 16922 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:17:24.761940 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49012.3us
I20260709 13:17:24.773566 16939 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:17:24.884789 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 143073us
I20260709 13:17:24.803445 16859 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.773527 16743 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.899199 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.897660 16491 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:24.888156 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 144942us
W20260709 13:17:24.899156 16989 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:17:24.904165 16491 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.900218 16953 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.906296 16491 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.900976 16939 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:24.907015 16930 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.913591 16922 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.927850 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.928675 16596 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 1: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:17:24.933830 16786 tablet_bootstrap.cc:654] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.934324 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:24.935184 16785 tablet_bootstrap.cc:492] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:24.942850 16785 ts_tablet_manager.cc:1403] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.346s	user 0.021s	sys 0.004s
I20260709 13:17:24.940734 16947 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.941249 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.941999 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.944572 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.945070 16916 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.946174 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.943385 16947 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.946820 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:24.929512 16922 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.952811 16787 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:24.957854 16787 ts_tablet_manager.cc:1403] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.435s	user 0.023s	sys 0.026s
I20260709 13:17:24.960791 16922 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:24.956830 16589 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 1: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:17:24.925729 16930 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.957706 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.968662 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.969854 16745 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.970253 16947 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.972886 16930 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:24.963335 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:24.977957 16948 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.969035 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:24.983626 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:24.984253 16743 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.984691 16745 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.989775 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:24.990520 16739 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:24.963886 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:24.991722 16860 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.992120 16860 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.992471 16860 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.993126 16785 raft_consensus.cc:359] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.993840 16785 raft_consensus.cc:385] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.994122 16785 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.995400 16787 raft_consensus.cc:359] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.996094 16787 raft_consensus.cc:385] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.996369 16787 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.997310 16743 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:24.996999 16787 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.998729 16787 ts_tablet_manager.cc:1434] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.003s
I20260709 13:17:24.955088 16939 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:24.999670 16787 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:24.994782 16785 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.001802 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:25.002420 16743 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:25.003222 16786 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:25.003652 16786 ts_tablet_manager.cc:1403] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.263s	user 0.018s	sys 0.028s
I20260709 13:17:25.006793 16739 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:25.008508 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:25.009435 16953 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.009909 16953 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.010710 16953 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.014391 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:25.015234 16739 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:25.016232 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:25.017270 16708 leader_election.cc:343] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.017833 16998 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.018368 16998 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.019961 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:25.019938 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:25.022313 16935 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.023725 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:25.026897 16785 ts_tablet_manager.cc:1434] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.000s
I20260709 13:17:25.028057 16947 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.028820 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:25.005862 16860 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.033649 16947 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.034511 16947 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.035423 16947 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.034653 16948 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.036082 16860 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:25.037899 16785 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:25.038981 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:25.028934 16745 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
W20260709 13:17:25.037528 16708 leader_election.cc:343] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.019321 16998 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:25.037699 16634 leader_election.cc:343] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.039065 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:25.042634 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800035460 after lost signal to thread 15479
I20260709 13:17:25.045809 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:25.046371 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:25.046856 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800035b40 after lost signal to thread 16528
I20260709 13:17:25.047165 16958 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.047469 16930 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.047837 16930 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:25.048002 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800245420 after lost signal to thread 16620
I20260709 13:17:25.046694 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:25.048190 16930 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.048919 16860 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:17:25.049023 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800245c60 after lost signal to thread 16695
W20260709 13:17:25.048163 16634 leader_election.cc:343] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.051855 16634 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:25.050843 16953 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:17:25.072649 16786 raft_consensus.cc:359] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.073170 16786 raft_consensus.cc:385] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.073419 16786 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
W20260709 13:17:25.066350 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800225d60 after lost signal to thread 16769
I20260709 13:17:25.069885 16939 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.078038 16786 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:25.077384 16708 leader_election.cc:343] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:25.079389 16634 leader_election.cc:343] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.068992 16859 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:17:25.079701 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:25.088658 16708 leader_election.cc:343] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.094478 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:25.092883 16815 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.098692 16786 ts_tablet_manager.cc:1434] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.000s
W20260709 13:17:25.098984 16708 leader_election.cc:343] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.103972 16786 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:25.109181 16958 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.126046 17004 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:25.126582 17004 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.103441 16634 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:25.137115 16787 tablet_bootstrap.cc:654] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.144345 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:25.145087 16663 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.146019 16785 tablet_bootstrap.cc:654] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.156308 17006 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:25.165536 16948 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.168115 16958 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.168744 17004 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.173216 17004 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:25.178224 15481 debug-util.cc:398] Leaking SignalData structure 0x7b080000f6e0 after lost signal to thread 16559
I20260709 13:17:25.182237 16930 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.183539 17006 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.184165 16930 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.184055 17006 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.185067 16786 tablet_bootstrap.cc:654] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.185709 17006 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:25.169657 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 84788.7us
I20260709 13:17:25.189455 16948 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:25.195297 16634 leader_election.cc:343] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:25.199265 16634 leader_election.cc:343] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.213342 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:25.214053 16739 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:25.215091 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:25.215701 16742 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.200402 16948 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.226126 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:25.227737 17007 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.228192 17007 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.228570 17007 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.231314 16787 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:25.211022 16930 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:17:25.232175 16787 ts_tablet_manager.cc:1403] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.233s	user 0.010s	sys 0.008s
I20260709 13:17:25.234335 16742 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:25.235875 16948 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.236006 16787 raft_consensus.cc:359] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.236770 16787 raft_consensus.cc:385] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.237082 16787 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.236825 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:25.224666 16634 leader_election.cc:343] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.241895 16939 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.241866 16947 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.242695 16939 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.211416 17006 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 13:17:25.244596 16669 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:25.244032 16939 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.245355 16947 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.247786 16785 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:25.248423 16785 ts_tablet_manager.cc:1403] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.211s	user 0.014s	sys 0.017s
I20260709 13:17:25.240581 16787 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:25.254352 17014 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:17:25.256050 16787 ts_tablet_manager.cc:1434] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:17:25.257398 16787 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:25.264089 16785 raft_consensus.cc:359] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.265228 16785 raft_consensus.cc:385] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.265578 16785 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.306039 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:25.308624 16595 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:17:25.309450 16595 pending_rounds.cc:85] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 0
I20260709 13:17:25.309863 16595 pending_rounds.cc:107] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:25.312389 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:25.312237 16785 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.316141 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:25.317862 16744 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:25.319855 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:25.321166 16745 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.327961 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:25.328532 16786 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
W20260709 13:17:25.329860 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080009a200 after lost signal to thread 15479
W20260709 13:17:25.331784 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08000c6280 after lost signal to thread 16528
W20260709 13:17:25.340888 16766 debug-util.cc:398] Leaking SignalData structure 0x7b0800249f60 after lost signal to thread 16620
W20260709 13:17:25.341372 16766 debug-util.cc:398] Leaking SignalData structure 0x7b0800249f80 after lost signal to thread 16621
I20260709 13:17:25.310361 16595 raft_consensus.cc:2889] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:17:25.342952 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024a7a0 after lost signal to thread 16695
W20260709 13:17:25.343333 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024a7c0 after lost signal to thread 16696
W20260709 13:17:25.344779 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024efe0 after lost signal to thread 16769
W20260709 13:17:25.345933 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024f240 after lost signal to thread 16911
W20260709 13:17:25.346683 16634 leader_election.cc:343] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.348259 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:25.352002 16815 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:25.354533 16785 ts_tablet_manager.cc:1434] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.106s	user 0.004s	sys 0.005s
I20260709 13:17:25.355769 16786 ts_tablet_manager.cc:1403] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.233s	user 0.027s	sys 0.024s
I20260709 13:17:25.351013 16663 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:25.368480 16785 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:25.365281 16863 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:17:25.361199 16922 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:17:25.372790 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:25.374055 16634 leader_election.cc:343] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.375051 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:25.377029 16494 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.378657 17007 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.379712 16494 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.386096 16494 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:17:25.388010 17004 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:25.388669 16863 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.397024 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:25.404940 16961 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.405725 16961 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.412055 17007 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.412185 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:25.412997 16961 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.417146 16786 raft_consensus.cc:359] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.418191 16786 raft_consensus.cc:385] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.418610 16786 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.419484 16786 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:25.421262 16634 leader_election.cc:343] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.422880 16786 ts_tablet_manager.cc:1434] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.005s
I20260709 13:17:25.430569 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:25.432958 16860 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.434698 16786 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:25.428651 16863 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.436668 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:25.437585 16743 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:25.438628 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:25.441103 16787 tablet_bootstrap.cc:654] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.444278 16739 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.456754 16785 tablet_bootstrap.cc:654] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.443553 16961 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.474677 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:25.472191 16860 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:25.496862 16787 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:25.498371 16860 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.517374 16961 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.542275 16745 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:25.543247 16871 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:25.544942 16634 leader_election.cc:343] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.546315 16786 tablet_bootstrap.cc:654] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.562220 16947 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:17:25.566891 16744 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:25.567757 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:25.569561 16939 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.570060 16939 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.565531 16739 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:25.571029 16939 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.582262 16859 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.583873 16859 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.584733 16787 ts_tablet_manager.cc:1403] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.328s	user 0.010s	sys 0.018s
W20260709 13:17:25.587352 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 100631us
I20260709 13:17:25.593890 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:25.594712 16859 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.595275 16859 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:25.596277 16859 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.591859 16947 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.616921 16787 raft_consensus.cc:359] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.634600 16787 raft_consensus.cc:385] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.635810 16787 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.637240 16786 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:25.637869 16786 ts_tablet_manager.cc:1403] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.210s	user 0.019s	sys 0.000s
I20260709 13:17:25.645100 16787 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.635886 16947 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.651042 16787 ts_tablet_manager.cc:1434] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.001s
I20260709 13:17:25.717790 16787 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
W20260709 13:17:25.650653 17029 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:17:25.656332 16591 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
W20260709 13:17:25.694638 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:25.658293 17004 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.725452 16591 pending_rounds.cc:85] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 0
I20260709 13:17:25.732172 17004 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.731981 16591 pending_rounds.cc:107] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:25.731266 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:25.735291 16947 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.743495 16591 raft_consensus.cc:2889] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:25.757877 17007 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.759865 16958 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.739931 16939 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:17:25.769759 16785 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:25.824579 16785 ts_tablet_manager.cc:1403] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.466s	user 0.017s	sys 0.036s
I20260709 13:17:25.790271 16786 raft_consensus.cc:359] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.825667 16786 raft_consensus.cc:385] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.826104 16786 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.811365 16494 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:17:25.805656 17040 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:17:25.790889 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:25.828970 16494 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:25.830225 16786 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.835094 16953 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:25.825793 17004 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:25.839653 16494 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:17:25.841670 16786 ts_tablet_manager.cc:1434] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.203s	user 0.014s	sys 0.009s
I20260709 13:17:25.841682 16494 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 2, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:17:25.844681 16786 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
W20260709 13:17:25.790584 16634 leader_election.cc:343] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.837239 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:25.795450 16859 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
W20260709 13:17:25.846112 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55576.9us
W20260709 13:17:25.861341 16634 leader_election.cc:343] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.874071 16953 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.878584 16787 tablet_bootstrap.cc:654] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.887306 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:25.888095 16744 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
W20260709 13:17:25.888984 16634 consensus_peers.cc:597] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:25.891417 16786 tablet_bootstrap.cc:654] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.907341 16491 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:17:25.933192 16787 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:25.932382 16493 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.933707 16787 ts_tablet_manager.cc:1403] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.216s	user 0.011s	sys 0.016s
I20260709 13:17:25.938529 16493 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.940515 16787 raft_consensus.cc:359] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.947255 16787 raft_consensus.cc:385] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:17:25.947616 16636 leader_election.cc:343] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.947609 16787 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.948130 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:25.948936 16786 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:25.949283 16939 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:25.948913 16787 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.949635 16786 ts_tablet_manager.cc:1403] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.105s	user 0.008s	sys 0.012s
I20260709 13:17:25.951067 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:25.951421 16787 ts_tablet_manager.cc:1434] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.014s	user 0.000s	sys 0.007s
I20260709 13:17:25.952225 17007 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.952675 16787 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:25.952625 17007 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.953176 17007 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.955127 16786 raft_consensus.cc:359] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.958226 16786 raft_consensus.cc:385] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.958596 16786 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.959502 16786 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.885520 16785 raft_consensus.cc:359] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.957552 16493 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.963099 16785 raft_consensus.cc:385] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:25.963416 16785 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:25.962893 16786 ts_tablet_manager.cc:1434] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:17:25.967167 16786 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:25.965921 16785 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.967573 16493 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 2, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.970626 16785 ts_tablet_manager.cc:1434] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.146s	user 0.002s	sys 0.005s
I20260709 13:17:25.971681 16785 tablet_bootstrap.cc:492] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:25.982564 17007 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.968413 16787 tablet_bootstrap.cc:654] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:25.987942 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:25.990629 16871 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:25.991976 16634 leader_election.cc:343] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:25.993130 16871 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:25.995230 16871 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:25.999169 17007 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.009312 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.009819 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:26.010012 16671 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:26.011555 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:26.012607 16871 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.013098 16871 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.013514 16871 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:26.016003 16708 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.017715 16786 tablet_bootstrap.cc:654] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.018384 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.017002 16744 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:26.025779 16708 leader_election.cc:343] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.029774 16787 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:26.030287 16787 ts_tablet_manager.cc:1403] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.078s	user 0.019s	sys 0.012s
I20260709 13:17:26.028265 16859 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:26.036381 16785 tablet_bootstrap.cc:654] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.042274 16871 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.047060 16744 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:26.051780 16787 raft_consensus.cc:359] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.052515 16787 raft_consensus.cc:385] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.052785 16787 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.053536 16787 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.055933 16787 ts_tablet_manager.cc:1434] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:17:26.058643 16787 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:26.060050 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:26.064118 16708 leader_election.cc:343] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.072134 16871 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.072714 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:26.073305 16671 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.075758 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.076865 16859 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.077379 16859 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:26.078166 16859 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:26.081933 16634 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.083366 16785 tablet_bootstrap.cc:492] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:26.083835 16785 ts_tablet_manager.cc:1403] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.112s	user 0.012s	sys 0.009s
I20260709 13:17:26.092464 16785 raft_consensus.cc:359] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.094213 16785 raft_consensus.cc:385] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.094558 16785 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.096539 16871 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:17:26.095451 16785 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.108917 16787 tablet_bootstrap.cc:654] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.112324 16785 ts_tablet_manager.cc:1434] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:17:26.116401 16786 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:26.116890 16786 ts_tablet_manager.cc:1403] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.000s
I20260709 13:17:26.117681 16785 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:26.118628 16871 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.118496 16671 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:26.144361 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.125433 16871 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.150940 16491 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:26.153004 17026 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.154742 17026 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:17:26.155414 16634 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.155658 17026 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.156594 17004 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.158545 16785 tablet_bootstrap.cc:654] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.161801 16871 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.162812 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.163458 16591 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:26.165750 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:26.166049 16787 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:26.166595 16787 ts_tablet_manager.cc:1403] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.110s	user 0.021s	sys 0.011s
I20260709 13:17:26.166531 16932 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.168452 16932 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.168828 16932 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.170557 16786 raft_consensus.cc:359] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.176236 16786 raft_consensus.cc:385] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.178076 16786 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.178246 16932 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.176468 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.172959 16491 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:26.177649 16787 raft_consensus.cc:359] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.181113 16787 raft_consensus.cc:385] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.181586 16787 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.180097 16786 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.179186 16669 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:26.196290 16786 ts_tablet_manager.cc:1434] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 13:17:26.194172 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:26.213169 16591 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.213617 16786 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:26.189152 16932 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.191565 16787 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.221784 16787 ts_tablet_manager.cc:1434] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.055s	user 0.001s	sys 0.009s
I20260709 13:17:26.226550 16787 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:26.224983 16591 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:26.227833 16785 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:26.228436 16785 ts_tablet_manager.cc:1403] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.115s	user 0.020s	sys 0.018s
I20260709 13:17:26.230847 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:26.231086 16785 raft_consensus.cc:359] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.231727 16785 raft_consensus.cc:385] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.232211 16785 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.233184 16787 tablet_bootstrap.cc:654] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.233817 16932 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.233525 16785 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.234337 16932 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:26.235249 16932 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.235980 16785 ts_tablet_manager.cc:1434] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:26.237115 16785 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:26.237861 16494 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.244558 16859 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.197463 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:26.245195 16859 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.245703 16671 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.247450 16859 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.253644 16785 tablet_bootstrap.cc:654] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.260707 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:26.262118 16634 leader_election.cc:343] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.262959 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:26.264051 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:26.273401 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.273898 16787 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:26.274364 16787 ts_tablet_manager.cc:1403] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.001s
I20260709 13:17:26.274488 16953 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.277235 16787 raft_consensus.cc:359] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.277926 16787 raft_consensus.cc:385] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.278509 16787 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.279191 16787 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.274895 16953 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.280534 16953 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.281466 16787 ts_tablet_manager.cc:1434] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:17:26.282491 16787 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:26.293327 16785 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:26.293764 16785 ts_tablet_manager.cc:1403] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.020s
I20260709 13:17:26.296835 16786 tablet_bootstrap.cc:654] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.297169 16787 tablet_bootstrap.cc:654] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.300283 16953 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.302343 16785 raft_consensus.cc:359] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.303000 16785 raft_consensus.cc:385] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.303274 16785 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.303980 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:26.303906 16785 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.304395 16493 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.305943 16785 ts_tablet_manager.cc:1434] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
W20260709 13:17:26.305898 16634 leader_election.cc:343] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.307101 17030 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.307562 17030 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.309475 16671 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:26.309607 17030 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.311494 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.313535 16785 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:26.323673 17026 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.324227 17026 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.324350 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:26.325568 16785 tablet_bootstrap.cc:654] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.326066 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:26.326517 17026 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.329018 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.330116 17030 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:26.335412 16671 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:26.336715 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:26.337389 17026 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.337384 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.337582 16953 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.337924 17026 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.338351 17026 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:26.339234 16708 leader_election.cc:343] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.340857 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:26.341463 16744 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.342793 16953 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.343282 16953 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.346283 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:26.347829 16636 leader_election.cc:343] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.349220 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.351326 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:26.353981 16787 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:26.354522 16787 ts_tablet_manager.cc:1403] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.072s	user 0.004s	sys 0.014s
I20260709 13:17:26.352576 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.354276 16953 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.355372 16859 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.356262 16859 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:26.357148 16859 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:26.355647 16634 leader_election.cc:343] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.360805 16634 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:26.361582 16939 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:26.362968 16787 raft_consensus.cc:359] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.362973 16786 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:26.367081 16787 raft_consensus.cc:385] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.367270 16786 ts_tablet_manager.cc:1403] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.154s	user 0.020s	sys 0.011s
I20260709 13:17:26.367431 16787 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.368295 16787 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.370425 16786 raft_consensus.cc:359] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.374917 16787 ts_tablet_manager.cc:1434] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:17:26.376154 16787 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:26.377328 16785 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:26.379632 16785 ts_tablet_manager.cc:1403] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.001s
I20260709 13:17:26.360023 17026 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.381985 17004 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.382462 17004 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.383811 17026 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.384609 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:26.385110 16671 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.385226 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:26.371114 16786 raft_consensus.cc:385] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.385901 16786 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
W20260709 13:17:26.386559 16708 leader_election.cc:343] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.390040 16671 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:26.391157 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.395537 16491 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:26.399207 16932 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.399910 16932 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:26.400951 16932 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.412921 17004 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.413192 16786 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.409755 16785 raft_consensus.cc:359] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.414419 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:26.414805 16785 raft_consensus.cc:385] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.415233 16785 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.415788 16786 ts_tablet_manager.cc:1434] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.048s	user 0.014s	sys 0.005s
I20260709 13:17:26.416092 16785 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:26.416625 16636 leader_election.cc:343] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.415668 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:26.419872 16634 leader_election.cc:343] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.420346 16634 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:26.422101 16859 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:26.422876 16786 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:26.422066 16787 tablet_bootstrap.cc:654] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.436784 16493 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.444597 17031 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:26.445079 17031 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.447116 16786 tablet_bootstrap.cc:654] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.447655 17031 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.449388 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:26.450906 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.451498 16671 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:26.455753 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:26.459064 17030 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.459467 17030 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:26.459820 17030 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.470765 16785 ts_tablet_manager.cc:1434] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.091s	user 0.005s	sys 0.003s
I20260709 13:17:26.483166 16744 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:17:26.488305 17030 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.491477 16785 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:26.496408 17030 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.497550 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:17:26.503059 16634 consensus_peers.cc:597] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.504007 16786 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:26.504505 16786 ts_tablet_manager.cc:1403] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.088s	user 0.020s	sys 0.013s
I20260709 13:17:26.507221 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:26.508023 16744 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:17:26.509891 16671 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:26.511127 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:26.512387 17070 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:26.512866 17070 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:26.513236 16787 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:26.513739 16787 ts_tablet_manager.cc:1403] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.138s	user 0.013s	sys 0.003s
I20260709 13:17:26.515043 16785 tablet_bootstrap.cc:654] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.513788 17070 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.516590 16787 raft_consensus.cc:359] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.517257 16787 raft_consensus.cc:385] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.517558 16787 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.519964 16787 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.537861 16787 ts_tablet_manager.cc:1434] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 13:17:26.521907 16786 raft_consensus.cc:359] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.547219 16786 raft_consensus.cc:385] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.547631 16786 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.547780 17026 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.548319 17026 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.550369 17026 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.552443 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.552856 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.553447 16671 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:26.553922 16708 leader_election.cc:343] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.554596 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.555294 16932 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.551424 16493 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:26.555625 16932 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.555970 16932 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.559559 16787 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:26.548477 16786 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.562330 16786 ts_tablet_manager.cc:1434] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 13:17:26.568919 16591 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.573858 16932 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.576648 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:26.577960 16708 leader_election.cc:343] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.578071 16786 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:26.584568 16786 tablet_bootstrap.cc:654] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.582453 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:26.591079 16932 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.591215 16671 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.585328 17026 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.591784 16591 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.568512 16787 tablet_bootstrap.cc:654] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.588835 16595 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.613336 16786 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:26.613857 16786 ts_tablet_manager.cc:1403] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.008s
I20260709 13:17:26.617815 16786 raft_consensus.cc:359] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.618675 16786 raft_consensus.cc:385] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.619036 16786 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.619802 16786 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.630244 16594 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.632647 16671 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:26.633234 16786 ts_tablet_manager.cc:1434] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:17:26.634449 16786 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:26.639804 16785 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:26.640271 16785 ts_tablet_manager.cc:1403] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.149s	user 0.014s	sys 0.031s
I20260709 13:17:26.641021 16786 tablet_bootstrap.cc:654] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.643468 17004 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.644990 16785 raft_consensus.cc:359] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.645653 16785 raft_consensus.cc:385] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.645927 16785 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.649984 16932 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:26.652060 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.655236 16932 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.655690 16932 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:26.653084 16785 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.656500 16932 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.665732 17004 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.668464 17004 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.670528 17004 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.671684 16785 ts_tablet_manager.cc:1434] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.002s
I20260709 13:17:26.678354 16785 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:26.681316 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:26.685971 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.686767 16745 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
W20260709 13:17:26.688015 16634 leader_election.cc:343] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.688762 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.690200 17004 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.690706 17004 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.691206 17004 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.715956 16787 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:26.718472 16787 ts_tablet_manager.cc:1403] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.159s	user 0.012s	sys 0.005s
I20260709 13:17:26.768646 16786 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:26.769162 16786 ts_tablet_manager.cc:1403] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.135s	user 0.018s	sys 0.005s
W20260709 13:17:26.771039 17075 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:17:26.772365 17004 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.773109 16785 tablet_bootstrap.cc:654] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:26.773730 17076 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.082s	user 0.004s	sys 0.000s
I20260709 13:17:26.802948 16871 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.803787 16786 raft_consensus.cc:359] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.804394 16871 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.804914 16786 raft_consensus.cc:385] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.805338 16786 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.806455 16786 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.813382 16787 raft_consensus.cc:359] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.814181 16787 raft_consensus.cc:385] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.814445 16787 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:26.815143 16787 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.816936 16787 ts_tablet_manager.cc:1434] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.098s	user 0.005s	sys 0.000s
I20260709 13:17:26.818022 16787 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:26.822381 16786 ts_tablet_manager.cc:1434] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.005s
I20260709 13:17:26.832657 16786 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:26.830417 16491 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.839176 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:26.840756 17004 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.861328 16744 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.884238 16786 tablet_bootstrap.cc:654] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:26.887661 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:26.899415 16871 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.899982 16871 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.902040 16871 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:26.905354 17007 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.906347 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.906839 17026 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.907368 17026 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.909381 17026 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:17:26.911273 16634 leader_election.cc:343] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.911796 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:26.912518 16669 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:26.915606 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:26.919478 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:26.918809 16708 leader_election.cc:343] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:26.920265 16671 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:17:26.920692 16744 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:26.923579 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.925107 17007 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:26.926165 17007 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:26.927404 17007 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.945267 16785 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:26.945786 16785 ts_tablet_manager.cc:1403] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.270s	user 0.014s	sys 0.014s
I20260709 13:17:26.951323 17004 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.951823 17004 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.951937 16785 raft_consensus.cc:359] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.952755 16785 raft_consensus.cc:385] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:26.953127 16785 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
W20260709 13:17:26.953949 16708 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.963933 16785 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.972332 17070 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.972959 17004 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.972837 17070 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:26.975311 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:26.975955 16739 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:26.976558 17070 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:26.977213 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:26.954695 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:26.978752 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.979270 16871 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:26.981379 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:26.982640 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:26.982231 17026 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.983599 16785 ts_tablet_manager.cc:1434] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.005s
I20260709 13:17:26.984438 17026 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.986461 17026 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.986545 16744 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:26.989594 16922 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:26.984722 16953 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:26.991012 16953 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.991335 16922 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:26.991734 16922 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:26.991371 16953 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:26.997988 16787 tablet_bootstrap.cc:654] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.012840 16785 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:27.023408 16786 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:27.023928 16786 ts_tablet_manager.cc:1403] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.196s	user 0.011s	sys 0.012s
I20260709 13:17:27.024147 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:27.024847 17030 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 13:17:27.042937 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:27.096598 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:27.096767 16786 raft_consensus.cc:359] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.097765 16786 raft_consensus.cc:385] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.098045 16786 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.098444 16953 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.099947 16786 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.102391 16922 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:27.109009 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 84819.7us
I20260709 13:17:27.111869 16591 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.112399 16786 ts_tablet_manager.cc:1434] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.088s	user 0.004s	sys 0.019s
I20260709 13:17:27.120031 16922 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:27.120333 16786 tablet_bootstrap.cc:492] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:27.115379 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:27.109681 17026 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.129851 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:27.131742 16663 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:27.134649 17026 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:27.136488 16953 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:27.140192 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:27.142169 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:27.143247 16669 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.143420 16587 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.143644 16785 tablet_bootstrap.cc:654] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.141060 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:27.146692 16663 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:17:27.155866 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:27.156509 17097 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:17:27.158404 16737 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:17:27.160353 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:27.161517 16744 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.160708 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:27.163332 16787 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:27.163924 16787 ts_tablet_manager.cc:1403] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.346s	user 0.018s	sys 0.011s
I20260709 13:17:27.167003 16491 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.169867 17095 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:27.178786 16587 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:27.179430 17095 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:17:27.163189 16634 leader_election.cc:343] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.180339 17095 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.182497 16785 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:27.184374 16785 ts_tablet_manager.cc:1403] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.174s	user 0.008s	sys 0.009s
I20260709 13:17:27.188010 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:27.236399 16744 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:27.237622 16671 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:27.238821 16922 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.263258 16922 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:27.264343 16922 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.341853 16786 tablet_bootstrap.cc:654] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.291926 16669 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:27.360172 16787 raft_consensus.cc:359] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.361022 16787 raft_consensus.cc:385] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.361294 16787 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.361917 16787 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.369220 16786 tablet_bootstrap.cc:492] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:27.369844 16786 ts_tablet_manager.cc:1403] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.250s	user 0.014s	sys 0.014s
I20260709 13:17:27.373304 16786 raft_consensus.cc:359] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.374169 16786 raft_consensus.cc:385] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.378480 16786 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.379170 16786 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.379659 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:27.381297 16634 leader_election.cc:343] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.380888 17087 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.385244 16787 ts_tablet_manager.cc:1434] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.221s	user 0.030s	sys 0.046s
I20260709 13:17:27.386322 16787 tablet_bootstrap.cc:492] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:27.392437 16785 raft_consensus.cc:359] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.393148 16785 raft_consensus.cc:385] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.393421 16785 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.394088 16785 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.399109 17070 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.404554 16871 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.409804 16786 ts_tablet_manager.cc:1434] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 13:17:27.380815 17031 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:17:27.414211 17031 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:27.414593 17031 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:27.414740 16595 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.420648 17031 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.421885 17031 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:27.428869 16708 consensus_peers.cc:597] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:27.430794 16785 ts_tablet_manager.cc:1434] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.246s	user 0.023s	sys 0.021s
W20260709 13:17:27.431999 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51104.1us
I20260709 13:17:27.453611 16785 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:27.455554 16786 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
W20260709 13:17:27.458879 17101 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.114s	user 0.001s	sys 0.002s
I20260709 13:17:27.466583 17087 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:27.501715 16932 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.504469 16932 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.517962 16787 tablet_bootstrap.cc:654] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:27.534821 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44034.9us
I20260709 13:17:27.545143 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:27.558111 16932 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:17:27.560963 16708 leader_election.cc:343] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.569955 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:27.525692 17004 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
W20260709 13:17:27.582396 16708 leader_election.cc:343] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.583053 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:27.584544 16922 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.582392 17087 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.587617 16932 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.587733 16786 tablet_bootstrap.cc:654] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.588112 16932 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:27.423060 17099 consensus_queue.cc:1175] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "170921ba08b74a33995ae7c7686f8109" responder_term: 3 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:27.599789 16491 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 2 to 3, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 3 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:27.609830 16491 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:27.627657 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:27.636775 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:27.639544 16708 leader_election.cc:343] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.656769 16932 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:27.659929 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:27.668450 16785 tablet_bootstrap.cc:654] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.678356 16787 tablet_bootstrap.cc:492] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:27.678906 16787 ts_tablet_manager.cc:1403] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.293s	user 0.012s	sys 0.031s
W20260709 13:17:27.680325 16708 leader_election.cc:343] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.681416 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:27.685242 16922 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.688520 16587 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:17:27.689040 16587 pending_rounds.cc:85] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 1
I20260709 13:17:27.689330 16587 pending_rounds.cc:107] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:17:27.689621 16587 raft_consensus.cc:2889] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:27.700901 16871 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:27.702257 16663 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:27.721414 16871 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:27.730391 16786 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:27.731004 16786 ts_tablet_manager.cc:1403] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.301s	user 0.018s	sys 0.011s
W20260709 13:17:27.733093 16697 tablet.cc:2406] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:27.742646 16786 raft_consensus.cc:359] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.743335 16786 raft_consensus.cc:385] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.743592 16786 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.744199 16786 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.750303 16786 ts_tablet_manager.cc:1434] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:17:27.751273 16786 tablet_bootstrap.cc:492] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:27.768637 16787 raft_consensus.cc:359] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.769333 16787 raft_consensus.cc:385] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.769611 16787 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.770227 16787 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.779604 16787 ts_tablet_manager.cc:1434] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.100s	user 0.007s	sys 0.000s
I20260709 13:17:27.789786 16787 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:27.790601 16493 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.793881 16785 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:27.796784 16785 ts_tablet_manager.cc:1403] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.343s	user 0.062s	sys 0.060s
I20260709 13:17:27.799443 16785 raft_consensus.cc:359] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.800172 16785 raft_consensus.cc:385] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.800486 16785 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.801131 17102 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:27.801157 16785 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.803659 16785 ts_tablet_manager.cc:1434] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:27.804493 16786 tablet_bootstrap.cc:654] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.804869 16785 tablet_bootstrap.cc:492] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:27.812471 16785 tablet_bootstrap.cc:654] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.844203 16786 tablet_bootstrap.cc:492] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:27.845148 16786 ts_tablet_manager.cc:1403] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.094s	user 0.010s	sys 0.007s
I20260709 13:17:27.846740 17105 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:27.848104 16786 raft_consensus.cc:359] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.848802 16786 raft_consensus.cc:385] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.849069 16786 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.849682 16786 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.852492 16786 ts_tablet_manager.cc:1434] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 13:17:27.852792 16785 tablet_bootstrap.cc:492] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:27.853390 16785 ts_tablet_manager.cc:1403] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.028s
I20260709 13:17:27.853571 16786 tablet_bootstrap.cc:492] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:27.858125 16785 raft_consensus.cc:359] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.859169 17113 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.860886 16787 tablet_bootstrap.cc:654] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.860816 17113 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.864843 16785 raft_consensus.cc:385] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.865715 16785 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.866281 16785 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.864450 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:27.870669 17031 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.871976 17031 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.872822 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:27.874450 17031 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:27.871832 17113 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:27.874454 16636 leader_election.cc:343] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.876160 16786 tablet_bootstrap.cc:654] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.876312 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:27.877004 16669 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:27.879004 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:27.879738 16739 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 0.
I20260709 13:17:27.880942 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:27.882031 17031 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.882393 17031 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.882706 17031 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:27.885150 16634 leader_election.cc:343] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.885581 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:27.885768 16785 ts_tablet_manager.cc:1434] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:17:27.890538 17087 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:27.905814 17031 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.908947 16785 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:27.914186 17031 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:27.917309 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:27.917896 16669 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.922293 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:27.922992 16739 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.933643 16739 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:27.934736 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:27.935454 17031 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:27.936156 17031 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:27.937562 16787 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:27.938022 16787 ts_tablet_manager.cc:1403] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.148s	user 0.010s	sys 0.010s
I20260709 13:17:27.939837 17031 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.940779 16669 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:27.944310 16787 raft_consensus.cc:359] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.944985 16787 raft_consensus.cc:385] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.945246 16787 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.947001 16871 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:27.947510 16871 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.953385 16871 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:27.954763 16786 tablet_bootstrap.cc:492] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:27.956472 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:27.957674 16785 tablet_bootstrap.cc:654] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.958176 16786 ts_tablet_manager.cc:1403] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.105s	user 0.026s	sys 0.012s
I20260709 13:17:27.958178 16669 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:27.961997 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:27.964246 17102 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:27.964648 17102 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:27.964489 16787 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.964985 17102 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:27.969250 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:27.969646 16786 raft_consensus.cc:359] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.970423 16786 raft_consensus.cc:385] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:27.970722 16786 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:27.972412 16787 ts_tablet_manager.cc:1434] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
I20260709 13:17:27.973032 16786 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:27.972756 16494 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:27.974732 16787 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:27.976082 17102 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:27.978087 16708 leader_election.cc:343] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:27.980756 16786 ts_tablet_manager.cc:1434] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:17:27.989087 16786 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:27.995004 16787 tablet_bootstrap.cc:654] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.995761 16786 tablet_bootstrap.cc:654] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:27.999365 17102 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:28.002667 17004 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.003156 17004 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.005081 17004 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.006574 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:28.007258 16739 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:28.009459 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:28.010890 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:28.012180 16785 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:28.012279 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:28.012621 16785 ts_tablet_manager.cc:1403] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.104s	user 0.008s	sys 0.004s
I20260709 13:17:28.012966 16671 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.016803 17087 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.017277 17087 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.017673 17087 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.017474 16785 raft_consensus.cc:359] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.018225 16785 raft_consensus.cc:385] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.018496 16785 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.019163 16785 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.021677 16785 ts_tablet_manager.cc:1434] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
W20260709 13:17:28.022451 16634 leader_election.cc:343] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.044400 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:28.053804 17004 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:17:28.055963 16708 leader_election.cc:343] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.061276 17106 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.061837 17106 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.064076 17106 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.065644 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:28.067914 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:28.069339 16785 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:28.070441 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:17:28.071475 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:28.072013 17031 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.075798 16785 tablet_bootstrap.cc:654] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.078099 16787 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:28.078574 16787 ts_tablet_manager.cc:1403] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.104s	user 0.011s	sys 0.033s
I20260709 13:17:28.082026 16787 raft_consensus.cc:359] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.082801 16787 raft_consensus.cc:385] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.083235 16787 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.084172 16787 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.086370 16787 ts_tablet_manager.cc:1434] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:28.093055 16785 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:28.093545 16785 ts_tablet_manager.cc:1403] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.001s
I20260709 13:17:28.095238 16787 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:28.096069 16785 raft_consensus.cc:359] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.097084 16786 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:28.097491 16786 ts_tablet_manager.cc:1403] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.109s	user 0.011s	sys 0.003s
I20260709 13:17:28.098165 16785 raft_consensus.cc:385] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.098410 16785 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.099421 17087 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.099543 16785 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.112879 16671 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:28.114060 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:28.117808 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:28.118618 16591 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.120966 17087 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.126088 16785 ts_tablet_manager.cc:1434] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 13:17:28.116925 16595 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.131978 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:28.132691 16742 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.137429 16786 raft_consensus.cc:359] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.116832 17105 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.138847 17105 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:28.139703 17105 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.138181 16786 raft_consensus.cc:385] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.147080 16786 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.147728 16786 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:28.149495 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:28.161768 16785 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:28.151130 17087 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.163295 16786 ts_tablet_manager.cc:1434] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.002s
I20260709 13:17:28.174706 16591 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:28.177909 16634 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:28.185659 16786 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:28.187399 16953 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.187919 16953 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:28.189056 16493 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:28.188802 16953 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.165858 17087 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.203773 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:28.204540 16742 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
W20260709 13:17:28.205006 16636 leader_election.cc:343] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.214911 16787 tablet_bootstrap.cc:654] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.210228 17087 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.221475 16786 tablet_bootstrap.cc:654] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.232734 16785 tablet_bootstrap.cc:654] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.234504 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:28.236121 16634 leader_election.cc:343] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.236534 16634 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:28.240363 17137 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.246948 16787 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:28.247488 16787 ts_tablet_manager.cc:1403] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.158s	user 0.009s	sys 0.003s
I20260709 13:17:28.247650 16491 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.249825 16786 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:28.250290 16786 ts_tablet_manager.cc:1403] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.008s
I20260709 13:17:28.250622 16787 raft_consensus.cc:359] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.251468 16787 raft_consensus.cc:385] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.251793 16787 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.252486 16787 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.253216 16786 raft_consensus.cc:359] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.253880 16786 raft_consensus.cc:385] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.254137 16786 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.255134 16787 ts_tablet_manager.cc:1434] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:28.254812 16786 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.256271 16787 tablet_bootstrap.cc:492] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:28.256500 16786 ts_tablet_manager.cc:1434] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:28.261241 16786 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:28.262657 16787 tablet_bootstrap.cc:654] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.277760 16786 tablet_bootstrap.cc:654] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.287706 16786 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:28.288169 16786 ts_tablet_manager.cc:1403] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 13:17:28.290704 16785 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:28.291265 16785 ts_tablet_manager.cc:1403] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.133s	user 0.014s	sys 0.000s
I20260709 13:17:28.291039 16786 raft_consensus.cc:359] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.291786 16786 raft_consensus.cc:385] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.292089 16786 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.292826 16786 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.294924 16786 ts_tablet_manager.cc:1434] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:17:28.296260 16786 tablet_bootstrap.cc:492] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:28.299863 16785 raft_consensus.cc:359] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.300616 16785 raft_consensus.cc:385] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.300891 16785 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.303639 16785 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.308182 16786 tablet_bootstrap.cc:654] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.309751 16785 ts_tablet_manager.cc:1434] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:17:28.310773 16785 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:28.313989 16953 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.314524 16953 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.316907 16953 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.317649 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:28.318519 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:28.319156 16634 leader_election.cc:343] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.319262 16742 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:28.321069 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:28.322041 16922 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.322566 16922 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.323280 17137 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.323684 17137 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.324043 17137 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.324913 16922 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:28.325492 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:28.326292 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:28.326701 16708 leader_election.cc:343] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:28.327836 16708 leader_election.cc:343] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.328238 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:28.328900 16787 tablet_bootstrap.cc:492] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:28.329274 16922 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.330897 16922 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.329433 16787 ts_tablet_manager.cc:1403] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.073s	user 0.035s	sys 0.014s
I20260709 13:17:28.331365 16922 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.334882 16787 raft_consensus.cc:359] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.337364 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:28.339578 16708 leader_election.cc:343] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.345417 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:28.341722 17137 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.335678 16787 raft_consensus.cc:385] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.346572 16787 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
W20260709 13:17:28.346779 16708 leader_election.cc:343] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.347007 16786 tablet_bootstrap.cc:492] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:28.347251 16708 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:28.347478 16786 ts_tablet_manager.cc:1403] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.000s
I20260709 13:17:28.348011 17102 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.347596 16787 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.348502 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:28.344394 16785 tablet_bootstrap.cc:654] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:28.349900 16634 leader_election.cc:343] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.350360 16787 ts_tablet_manager.cc:1434] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:17:28.351436 16787 tablet_bootstrap.cc:492] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:28.355217 16786 raft_consensus.cc:359] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.357578 16786 raft_consensus.cc:385] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.357933 16786 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.344904 16922 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:28.359990 17102 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.360467 17102 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.362285 16787 tablet_bootstrap.cc:654] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.363663 16786 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.364396 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:28.364998 16671 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:28.366209 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:28.366421 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:28.362411 17102 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:28.367996 16922 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:28.368474 16922 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:28.367704 16708 leader_election.cc:343] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.370151 16922 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.380069 16786 ts_tablet_manager.cc:1434] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:17:28.380261 17137 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.381099 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:28.381357 16786 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:28.381698 16742 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.391640 16785 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:28.392042 16786 tablet_bootstrap.cc:654] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.392376 16785 ts_tablet_manager.cc:1403] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.009s
I20260709 13:17:28.402314 16742 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:28.403179 16922 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.404275 16785 raft_consensus.cc:359] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.404374 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:28.406908 17031 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.407464 16953 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.407344 17031 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.407943 16953 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:28.408761 16953 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.409338 17031 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.410312 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:28.410709 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:28.412442 16561 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:28.405375 16922 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:28.406215 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:28.406903 16785 raft_consensus.cc:385] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.413342 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:28.414141 16785 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.414414 16671 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.416163 17031 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:28.417899 16708 leader_election.cc:343] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.417677 16785 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.420159 16785 ts_tablet_manager.cc:1434] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:17:28.421180 16785 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:28.435556 16671 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:28.437541 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:28.445070 17105 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:28.446010 16787 tablet_bootstrap.cc:492] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:28.448036 16787 ts_tablet_manager.cc:1403] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.012s
I20260709 13:17:28.450836 17105 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:28.451094 16787 raft_consensus.cc:359] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.451784 17105 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.452445 16786 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:28.453048 16786 ts_tablet_manager.cc:1403] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.072s	user 0.014s	sys 0.008s
I20260709 13:17:28.451763 16787 raft_consensus.cc:385] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.453756 16787 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.456487 16787 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.456732 16494 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:28.458266 16596 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.459800 16787 ts_tablet_manager.cc:1434] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:17:28.461424 16787 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:28.484159 16785 tablet_bootstrap.cc:654] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.485227 16786 raft_consensus.cc:359] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.486158 16786 raft_consensus.cc:385] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.486490 16786 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.490882 16786 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.496543 16786 ts_tablet_manager.cc:1434] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 13:17:28.496757 16787 tablet_bootstrap.cc:654] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.496356 17031 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:28.499575 16786 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:28.509200 16493 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:28.529783 16708 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:28.540583 17031 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.541366 16786 tablet_bootstrap.cc:654] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:28.541117 17031 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.555792 16596 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:28.572425 17031 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.563048 16595 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:28.600826 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57235us
I20260709 13:17:28.639478 16787 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
W20260709 13:17:28.639361 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58327.8us
I20260709 13:17:28.640522 16871 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.640781 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:28.641003 16871 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.640313 16787 ts_tablet_manager.cc:1403] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.179s	user 0.024s	sys 0.041s
I20260709 13:17:28.647935 16953 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.658267 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:28.658092 16953 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.662709 16787 raft_consensus.cc:359] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.663829 16787 raft_consensus.cc:385] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.664146 16787 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.664739 16787 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.667765 16871 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:28.660218 17106 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:17:28.663046 16953 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:28.661852 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:28.679376 16787 ts_tablet_manager.cc:1434] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:17:28.682839 16786 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:28.687366 16786 ts_tablet_manager.cc:1403] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.188s	user 0.016s	sys 0.006s
I20260709 13:17:28.684512 16785 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:28.684844 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:28.688028 16785 ts_tablet_manager.cc:1403] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.267s	user 0.014s	sys 0.032s
I20260709 13:17:28.701732 16787 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:28.703071 16785 raft_consensus.cc:359] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.703814 16785 raft_consensus.cc:385] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.704085 16785 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.704864 16785 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.712160 16561 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:17:28.721841 16634 leader_election.cc:343] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:28.727912 16785 ts_tablet_manager.cc:1434] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.040s	user 0.000s	sys 0.007s
I20260709 13:17:28.736327 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:28.740713 16785 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:28.739509 16786 raft_consensus.cc:359] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.766300 16786 raft_consensus.cc:385] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:28.766722 16786 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:28.738395 17087 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.718397 17090 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
W20260709 13:17:28.795713 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56386.7us
I20260709 13:17:28.744453 16744 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:28.911626 17161 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.912416 17161 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:28.929684 17158 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:17:28.846074 17139 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.859331 17156 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:28.845436 17109 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.104s
I20260709 13:17:28.828948 17087 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.985999 16786 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:28.995167 16786 ts_tablet_manager.cc:1434] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.307s	user 0.034s	sys 0.035s
I20260709 13:17:29.006155 17161 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:29.015130 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:29.013643 17102 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:17:28.944470 17139 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.034626 17139 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:29.046183 17087 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:29.056865 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46909.7us
W20260709 13:17:29.057964 16708 leader_election.cc:343] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:29.061200 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 349149us
I20260709 13:17:29.066623 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:29.067185 16708 leader_election.cc:343] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.075286 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.069216 16785 tablet_bootstrap.cc:654] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.076138 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:29.077174 17161 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.078464 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:29.070223 17163 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.088s	user 0.000s	sys 0.004s
I20260709 13:17:29.068643 16593 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:29.085582 16634 leader_election.cc:343] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.091550 16787 tablet_bootstrap.cc:654] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:29.094156 16634 leader_election.cc:343] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:29.144236 16708 leader_election.cc:343] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:29.161252 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 425824us
I20260709 13:17:29.146176 17105 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:17:29.172065 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.213634 16786 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:29.177088 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:29.235661 16785 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:29.240823 16785 ts_tablet_manager.cc:1403] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.501s	user 0.016s	sys 0.036s
I20260709 13:17:29.187240 17169 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.249276 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:29.263705 16636 leader_election.cc:343] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:29.280596 16708 consensus_peers.cc:597] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:29.264451 16636 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:17:29.309904 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46261.7us
I20260709 13:17:29.247424 17169 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.312170 16953 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.306066 17105 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.331579 16787 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:29.332144 16787 ts_tablet_manager.cc:1403] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.631s	user 0.060s	sys 0.081s
W20260709 13:17:29.333107 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 97901.1us
I20260709 13:17:29.319133 17169 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.384637 16785 raft_consensus.cc:359] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.385393 16785 raft_consensus.cc:385] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.385671 16785 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.389294 16786 tablet_bootstrap.cc:654] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.386301 16785 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.336942 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:29.332763 17105 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.404417 16736 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:29.404973 17105 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:29.414570 16785 ts_tablet_manager.cc:1434] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.173s	user 0.008s	sys 0.000s
W20260709 13:17:29.421339 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:29.421216 16708 leader_election.cc:343] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.432292 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.407516 17174 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:29.443805 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:29.437894 17179 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.456602 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:29.457779 16708 leader_election.cc:343] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.462682 17087 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.463361 17087 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.463791 17087 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.468968 16787 raft_consensus.cc:359] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.469524 16787 raft_consensus.cc:385] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.469791 16787 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.471256 16787 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.478273 16787 ts_tablet_manager.cc:1434] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.146s	user 0.005s	sys 0.002s
I20260709 13:17:29.493881 16785 tablet_bootstrap.cc:492] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:29.486442 17087 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.508885 17087 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:29.510047 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:29.514765 16664 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:29.509790 17169 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.512394 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:29.513906 16787 tablet_bootstrap.cc:492] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:29.519104 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.522601 16738 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.525497 17171 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.525893 17171 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.526204 17171 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.552529 16786 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:29.553263 16786 ts_tablet_manager.cc:1403] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.340s	user 0.016s	sys 0.008s
I20260709 13:17:29.556895 16785 tablet_bootstrap.cc:654] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.557565 17160 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.558095 17160 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.560715 17160 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:29.563194 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:29.564527 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:29.572054 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:29.572690 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:29.575544 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:29.583180 17182 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:29.593518 16787 tablet_bootstrap.cc:654] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.597424 16738 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:29.600046 17169 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:29.617923 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:29.640275 16786 raft_consensus.cc:359] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.641038 16786 raft_consensus.cc:385] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.641330 16786 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.641947 16786 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.646593 17169 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.647188 17169 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:29.648015 17169 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:29.649463 16634 leader_election.cc:343] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.650815 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:29.653483 16738 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.660661 16786 ts_tablet_manager.cc:1434] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.107s	user 0.008s	sys 0.000s
I20260709 13:17:29.661659 16786 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:29.669407 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:29.670817 17171 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.673743 16589 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.705046 16785 tablet_bootstrap.cc:492] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:29.705624 16785 ts_tablet_manager.cc:1403] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.212s	user 0.022s	sys 0.006s
I20260709 13:17:29.705916 16787 tablet_bootstrap.cc:492] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:29.706354 16787 ts_tablet_manager.cc:1403] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.193s	user 0.014s	sys 0.014s
W20260709 13:17:29.713737 16634 leader_election.cc:343] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:29.741998 17194 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:17:29.747555 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:29.748539 16787 raft_consensus.cc:359] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.749302 16787 raft_consensus.cc:385] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.749573 16787 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.750262 16787 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.752076 16787 ts_tablet_manager.cc:1434] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.001s
I20260709 13:17:29.753291 16787 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:29.758936 16785 raft_consensus.cc:359] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.759559 16785 raft_consensus.cc:385] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.759824 16785 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.760440 16785 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.763832 16786 tablet_bootstrap.cc:654] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:29.767424 16708 leader_election.cc:343] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.774135 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:29.774755 16665 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.775281 16738 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:29.776610 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.778193 17087 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.778674 17087 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:29.779690 17087 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.783607 16785 ts_tablet_manager.cc:1434] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.078s	user 0.002s	sys 0.004s
I20260709 13:17:29.791887 16785 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:29.782392 17171 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:29.801388 16493 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:29.806315 16786 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:29.806799 16786 ts_tablet_manager.cc:1403] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.145s	user 0.014s	sys 0.007s
I20260709 13:17:29.808841 16665 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:29.810281 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.813681 17171 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.814185 17171 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:29.815025 17171 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.820255 16787 tablet_bootstrap.cc:654] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.822762 16786 raft_consensus.cc:359] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.826284 16786 raft_consensus.cc:385] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.826941 16786 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.828557 16786 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.836033 16786 ts_tablet_manager.cc:1434] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:17:29.840580 16786 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:29.853925 16785 tablet_bootstrap.cc:654] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.864866 16786 tablet_bootstrap.cc:654] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.868541 16787 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:29.869118 16787 ts_tablet_manager.cc:1403] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.002s
I20260709 13:17:29.886226 16787 raft_consensus.cc:359] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.891463 16787 raft_consensus.cc:385] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.892119 16787 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.893252 16785 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:29.892889 16787 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.895066 16785 ts_tablet_manager.cc:1403] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.103s	user 0.018s	sys 0.006s
I20260709 13:17:29.896581 16787 ts_tablet_manager.cc:1434] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:17:29.899926 16787 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:29.902408 16785 raft_consensus.cc:359] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.907199 16785 raft_consensus.cc:385] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.907672 16785 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.910884 16786 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:29.912132 16786 ts_tablet_manager.cc:1403] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.000s
I20260709 13:17:29.912400 17087 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:29.912920 17087 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.914300 16786 raft_consensus.cc:359] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.914784 16786 raft_consensus.cc:385] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.915221 16786 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.915879 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:29.916239 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:29.915894 16786 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.916956 16738 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
W20260709 13:17:29.917232 16634 leader_election.cc:343] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.918017 17087 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:29.918223 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.919027 17087 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:29.919230 16786 ts_tablet_manager.cc:1434] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:29.919456 17087 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:29.919876 17087 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.920245 16786 tablet_bootstrap.cc:492] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:29.924070 16785 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.925761 16785 ts_tablet_manager.cc:1434] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.000s
I20260709 13:17:29.926502 16785 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:29.927342 17087 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.929853 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:29.931094 16634 leader_election.cc:343] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:29.931890 16787 tablet_bootstrap.cc:654] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.932194 16785 tablet_bootstrap.cc:654] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.935513 17087 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:29.939124 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:29.939819 16738 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:29.941571 16787 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:29.942080 16787 ts_tablet_manager.cc:1403] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.006s
I20260709 13:17:29.944267 16786 tablet_bootstrap.cc:654] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.945767 16787 raft_consensus.cc:359] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.946463 16787 raft_consensus.cc:385] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.946830 16787 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.947630 16787 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.948169 16785 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:29.948786 16785 ts_tablet_manager.cc:1403] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.011s
I20260709 13:17:29.949817 16787 ts_tablet_manager.cc:1434] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:17:29.950907 16787 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:29.951694 16785 raft_consensus.cc:359] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.952463 16785 raft_consensus.cc:385] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:29.952790 16785 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:29.953738 16785 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.949976 16738 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:29.956096 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:29.956678 16785 ts_tablet_manager.cc:1434] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:29.957368 17087 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:29.957882 16785 tablet_bootstrap.cc:492] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:29.958065 16787 tablet_bootstrap.cc:654] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:29.958109 17087 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:29.960199 17087 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:29.966018 16786 tablet_bootstrap.cc:492] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:29.966619 16786 ts_tablet_manager.cc:1403] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.047s	user 0.014s	sys 0.001s
I20260709 13:17:30.007023 17162 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.007702 17162 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.010433 17162 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.015899 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.016783 17172 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.017046 16742 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:30.018003 17169 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.018472 17169 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.017494 17172 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.020605 17169 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.022912 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:30.058210 17172 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:30.060992 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.069764 16742 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:30.063459 17179 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.070781 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.071066 17179 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.077785 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:30.065240 16634 leader_election.cc:343] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.072026 16787 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:30.072239 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:30.079139 16787 ts_tablet_manager.cc:1403] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.128s	user 0.022s	sys 0.001s
I20260709 13:17:30.080344 17162 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.073781 16785 tablet_bootstrap.cc:654] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.082652 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:30.083467 17109 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.083895 17109 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.083583 16787 raft_consensus.cc:359] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.084273 17109 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.084374 16787 raft_consensus.cc:385] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.084734 16787 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.085366 16787 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.088919 17162 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.093191 16787 ts_tablet_manager.cc:1434] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:17:30.092828 17109 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.094151 16787 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:30.094305 16922 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.094753 16922 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.095492 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:30.096593 16922 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:17:30.096735 16634 leader_election.cc:343] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.064755 16786 raft_consensus.cc:359] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.098280 16786 raft_consensus.cc:385] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.100343 16786 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.101897 16786 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.104473 16787 tablet_bootstrap.cc:654] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:30.105962 16708 leader_election.cc:343] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.099972 17179 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:30.100939 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.107448 16664 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:30.109087 16786 ts_tablet_manager.cc:1434] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.142s	user 0.010s	sys 0.015s
I20260709 13:17:30.113014 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:30.115365 16922 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.115839 16922 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.116269 16922 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.117431 16785 tablet_bootstrap.cc:492] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:30.117890 16785 ts_tablet_manager.cc:1403] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.160s	user 0.019s	sys 0.017s
I20260709 13:17:30.118770 17162 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.119277 17162 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.122015 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.124560 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:30.125270 16493 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.126895 17169 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.129076 16785 raft_consensus.cc:359] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.129776 16785 raft_consensus.cc:385] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.130039 16785 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.131383 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:30.132867 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.133153 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:30.133522 16665 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:30.133689 16738 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.136883 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.133744 16786 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:30.140720 17162 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.142220 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.143203 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:30.143720 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:30.145054 17162 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.144557 17179 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.145869 17179 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.146453 17179 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.147794 17109 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:30.150319 16708 leader_election.cc:343] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.130692 16785 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:30.155489 16708 leader_election.cc:343] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.156334 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:30.157902 16708 leader_election.cc:343] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.151527 16922 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.156018 16493 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.160171 17172 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.168119 17162 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.168846 17162 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.168018 16738 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:30.183173 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:30.184322 16922 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:30.184762 17173 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.186408 17173 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:30.187397 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.201280 16785 ts_tablet_manager.cc:1434] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 13:17:30.213644 16785 tablet_bootstrap.cc:492] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:30.215587 16491 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:30.215394 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:30.228272 16669 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.229333 17179 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.220906 17162 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.231945 17179 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:30.222864 17173 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.237588 17169 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:30.237865 17193 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.238382 17193 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.241282 17193 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.247159 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:30.247159 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.256920 16559 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:30.283295 16787 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:30.283972 16787 ts_tablet_manager.cc:1403] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.190s	user 0.038s	sys 0.068s
I20260709 13:17:30.262873 17182 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.285408 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.287720 16739 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:30.291234 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:30.295780 16708 leader_election.cc:343] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.284979 17164 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.294497 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:30.302414 16786 tablet_bootstrap.cc:654] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.301749 17169 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:30.308667 16595 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.305362 17169 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:30.312526 17169 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.305518 17179 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:17:30.314635 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:30.303844 17164 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.315336 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:30.316732 17164 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:30.302910 16589 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.325708 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.307770 16665 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.337378 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:30.340744 16634 leader_election.cc:343] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.347800 17169 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.354694 16787 raft_consensus.cc:359] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.363516 16787 raft_consensus.cc:385] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.363828 16787 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.364627 17169 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.364518 16787 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.351447 16665 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:30.351344 16493 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.353330 16785 tablet_bootstrap.cc:654] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:30.359702 16708 leader_election.cc:343] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:30.372640 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:30.367762 16787 ts_tablet_manager.cc:1434] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.000s
I20260709 13:17:30.333051 16669 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
W20260709 13:17:30.376549 16708 leader_election.cc:343] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:30.378278 16708 leader_election.cc:343] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.378656 16787 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:30.378906 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:30.380452 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:30.380976 16745 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.384148 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:30.387954 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:30.384895 17223 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.388911 17161 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:30.417444 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:30.379788 17193 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 13:17:30.428848 16634 leader_election.cc:343] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.429584 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:30.434788 17164 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.435338 17164 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:30.436180 17164 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.439304 17087 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.439800 17087 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.441854 17087 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.450675 16786 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:30.451376 16786 ts_tablet_manager.cc:1403] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.318s	user 0.016s	sys 0.015s
I20260709 13:17:30.453203 16745 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:30.420747 17161 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:30.456945 17167 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:17:30.475124 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:30.484367 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:30.488827 16634 leader_election.cc:343] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.481403 17161 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.501890 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.499541 17169 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:30.496399 16591 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:30.518266 17169 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:30.519435 17169 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.543315 17218 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.544111 17218 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.564616 17109 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:30.565207 17109 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.569624 16786 raft_consensus.cc:359] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.570415 16786 raft_consensus.cc:385] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.570734 16786 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.571532 16786 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.575513 16787 tablet_bootstrap.cc:654] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:30.578711 16636 leader_election.cc:343] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.579252 16785 tablet_bootstrap.cc:492] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:30.579840 16785 ts_tablet_manager.cc:1403] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.368s	user 0.016s	sys 0.016s
I20260709 13:17:30.579245 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:30.581166 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:30.584784 16786 ts_tablet_manager.cc:1434] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.133s	user 0.018s	sys 0.007s
I20260709 13:17:30.570189 17218 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.591013 17109 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:30.592867 17167 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:30.592726 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:30.594211 16634 leader_election.cc:343] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.619145 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:30.627034 16636 leader_election.cc:343] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:30.627609 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:30.630846 17087 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:30.767490 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 137847us
I20260709 13:17:30.772122 16787 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:30.772637 16787 ts_tablet_manager.cc:1403] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.395s	user 0.020s	sys 0.036s
I20260709 13:17:30.789649 16785 raft_consensus.cc:359] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.790449 16785 raft_consensus.cc:385] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.790717 16785 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.791889 16785 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.794483 16785 ts_tablet_manager.cc:1434] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.214s	user 0.011s	sys 0.004s
I20260709 13:17:30.797223 16785 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:30.805383 16786 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:30.806203 16787 raft_consensus.cc:359] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.818145 16787 raft_consensus.cc:385] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:30.818508 16787 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
W20260709 13:17:30.808151 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:30.830711 16787 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:30.869098 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:30.871272 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:17:30.885962 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 88220.8us
W20260709 13:17:30.932063 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 59560.3us
I20260709 13:17:30.932551 16785 tablet_bootstrap.cc:654] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:30.921038 17182 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:30.979322 16787 ts_tablet_manager.cc:1434] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.206s	user 0.037s	sys 0.017s
W20260709 13:17:30.983464 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 75694.8us
I20260709 13:17:30.999800 17161 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.009210 17161 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.015259 16786 tablet_bootstrap.cc:654] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.027954 16787 tablet_bootstrap.cc:492] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:31.077589 16591 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:31.084211 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 86979.8us
W20260709 13:17:31.138428 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54664.1us
I20260709 13:17:31.116673 17161 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:31.171757 17248 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.172364 17248 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.174151 16787 tablet_bootstrap.cc:654] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.181169 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:31.194160 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:31.196318 17248 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:31.204800 17245 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:31.202243 16493 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.206799 16667 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:31.216536 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:31.217407 16745 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:17:31.217828 17243 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:31.222296 17223 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:31.226137 16634 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:31.228204 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:31.229127 17171 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:31.229481 17171 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.229820 17171 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:31.252941 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:31.253911 16708 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:31.254850 16708 leader_election.cc:343] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.260037 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:31.267617 17105 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:17:31.287868 16786 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:31.288383 16786 ts_tablet_manager.cc:1403] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.483s	user 0.009s	sys 0.020s
I20260709 13:17:31.295518 16785 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:31.296057 16785 ts_tablet_manager.cc:1403] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.499s	user 0.017s	sys 0.000s
I20260709 13:17:31.303893 16785 raft_consensus.cc:359] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.304646 16785 raft_consensus.cc:385] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.304939 16785 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.305608 16785 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.307641 16786 raft_consensus.cc:359] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.308331 16786 raft_consensus.cc:385] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.308599 16786 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.309186 16786 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.311460 16786 ts_tablet_manager.cc:1434] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:17:31.323112 16786 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:31.325067 16785 ts_tablet_manager.cc:1434] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:17:31.329584 16785 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:31.346588 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:17:31.341804 17241 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.356261 17162 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:31.381382 17258 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:31.382542 16785 tablet_bootstrap.cc:654] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.368928 17162 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:31.363627 17257 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 13:17:31.392083 16787 tablet_bootstrap.cc:492] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:31.392524 16787 ts_tablet_manager.cc:1403] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.365s	user 0.031s	sys 0.031s
I20260709 13:17:31.366552 17241 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.392405 17162 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.396795 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:31.397953 16634 leader_election.cc:343] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.403375 17171 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.405951 17171 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:31.406373 16787 raft_consensus.cc:359] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.412129 16787 raft_consensus.cc:385] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.412494 16787 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.417821 17241 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:31.413151 16787 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.424892 16787 ts_tablet_manager.cc:1434] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.002s
I20260709 13:17:31.430821 16786 tablet_bootstrap.cc:654] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.431559 16787 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:31.431555 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:31.443498 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:31.444410 16666 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.435708 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:17:31.470296 16708 leader_election.cc:343] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.481959 17162 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.496695 17109 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.497206 17109 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.498011 17162 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:31.492924 16785 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:31.499085 16785 ts_tablet_manager.cc:1403] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.170s	user 0.021s	sys 0.011s
I20260709 13:17:31.499414 16588 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:31.500648 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:31.503167 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:31.504834 17109 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:31.504834 16634 leader_election.cc:343] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.506659 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:31.506933 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:31.509464 17175 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:31.515501 16787 tablet_bootstrap.cc:654] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.516819 16786 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:31.517333 16786 ts_tablet_manager.cc:1403] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.199s	user 0.011s	sys 0.006s
I20260709 13:17:31.517127 16785 raft_consensus.cc:359] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.517807 16785 raft_consensus.cc:385] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.518111 16785 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.518258 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:31.520411 16786 raft_consensus.cc:359] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.524690 16786 raft_consensus.cc:385] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.525602 16786 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
W20260709 13:17:31.522997 16636 leader_election.cc:343] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.527711 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:31.527829 16786 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.528708 17247 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.529735 16786 ts_tablet_manager.cc:1434] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.001s
I20260709 13:17:31.531026 16786 tablet_bootstrap.cc:492] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:31.534574 16666 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:31.526983 17245 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:31.559118 17245 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:31.559970 17245 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.561287 17245 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.537467 16786 tablet_bootstrap.cc:654] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:31.567927 16636 leader_election.cc:343] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.568382 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:31.568589 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:31.518786 16785 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.549465 17182 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.521098 17162 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:31.550787 16494 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:31.573557 17109 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.579537 16785 ts_tablet_manager.cc:1434] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.000s
I20260709 13:17:31.580646 16785 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:31.572575 17182 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.575362 17233 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:31.596352 17233 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:31.597258 17233 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.607136 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:31.630364 16785 tablet_bootstrap.cc:654] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.633445 17182 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:31.634657 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:31.636354 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:31.638707 17221 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.640264 16787 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:31.640740 16787 ts_tablet_manager.cc:1403] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.209s	user 0.035s	sys 0.033s
I20260709 13:17:31.645313 16787 raft_consensus.cc:359] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.647847 16787 raft_consensus.cc:385] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.648074 16787 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.648658 16787 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.650846 16787 ts_tablet_manager.cc:1434] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.001s
I20260709 13:17:31.657168 16588 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.660064 16786 tablet_bootstrap.cc:492] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:31.660580 16786 ts_tablet_manager.cc:1403] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.130s	user 0.022s	sys 0.011s
I20260709 13:17:31.661360 16787 tablet_bootstrap.cc:492] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:31.667703 16786 raft_consensus.cc:359] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.668403 16786 raft_consensus.cc:385] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.668668 16786 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.671032 16786 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.678160 16785 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:31.678819 16785 ts_tablet_manager.cc:1403] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.003s
I20260709 13:17:31.710305 16787 tablet_bootstrap.cc:654] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:31.709486 16785 raft_consensus.cc:359] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.711369 16785 raft_consensus.cc:385] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:31.711654 16785 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:31.714107 16786 ts_tablet_manager.cc:1434] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.053s	user 0.002s	sys 0.006s
I20260709 13:17:31.710657 16494 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:31.717111 16494 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:31.725411 16785 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.720722 17167 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.736320 16786 tablet_bootstrap.cc:492] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:31.726516 17171 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.729328 17167 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.749329 17171 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.751629 17171 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:31.726439 17223 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.766820 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:17:31.769176 16708 leader_election.cc:343] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.780684 16785 ts_tablet_manager.cc:1434] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.102s	user 0.003s	sys 0.004s
I20260709 13:17:31.781800 16785 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:31.773898 17223 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:31.787716 17167 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:31.786984 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:31.785388 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:31.803357 16591 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.812125 17223 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:31.794116 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:31.813930 16636 leader_election.cc:343] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.816919 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:31.827149 16670 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:31.828826 16708 leader_election.cc:343] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.837688 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:31.838438 16666 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
W20260709 13:17:31.843677 16634 leader_election.cc:343] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:31.844167 16634 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:31.845993 17245 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:31.849388 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:31.853843 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:31.854656 17233 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:31.855062 17233 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.855423 17233 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.859648 17105 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:31.860185 17105 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:31.860611 17105 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:31.918883 17164 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:31.922683 17164 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.038532 16786 tablet_bootstrap.cc:654] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.038578 17221 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:32.062834 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52643.8us
I20260709 13:17:32.039211 17221 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.067963 16785 tablet_bootstrap.cc:700] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent opening tablet: real 0.174s	user 0.018s	sys 0.068s
I20260709 13:17:32.068861 16785 tablet_bootstrap.cc:654] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:32.085395 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:32.106348 16787 tablet_bootstrap.cc:492] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:32.106838 16787 ts_tablet_manager.cc:1403] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.446s	user 0.024s	sys 0.030s
I20260709 13:17:32.107679 17233 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.109573 17233 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.156124 16786 tablet_bootstrap.cc:492] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:32.156823 16786 ts_tablet_manager.cc:1403] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.434s	user 0.014s	sys 0.004s
I20260709 13:17:32.191416 16787 raft_consensus.cc:359] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.192142 16787 raft_consensus.cc:385] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.192411 16787 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.192761 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:32.193347 16665 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.193027 16787 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.197090 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.203948 17221 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:32.206199 16787 ts_tablet_manager.cc:1434] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.099s	user 0.003s	sys 0.003s
I20260709 13:17:32.207356 16787 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:32.176091 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:32.226327 16786 raft_consensus.cc:359] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.227393 16786 raft_consensus.cc:385] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.227728 16786 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.229064 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:32.229710 17105 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.214818 17164 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.231837 17105 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.237855 16665 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
W20260709 13:17:32.252632 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85298.3us
I20260709 13:17:32.245005 16786 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.273634 16786 ts_tablet_manager.cc:1434] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.116s	user 0.011s	sys 0.000s
I20260709 13:17:32.275177 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:32.278481 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:32.278533 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:32.317960 16662 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:32.284315 16708 leader_election.cc:343] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.287647 16589 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.296766 16787 tablet_bootstrap.cc:654] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:32.328491 16708 leader_election.cc:343] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.333989 16662 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:32.338006 16786 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
W20260709 13:17:32.340030 16708 consensus_peers.cc:597] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:32.345328 16708 leader_election.cc:343] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.348608 16785 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:32.349076 16785 ts_tablet_manager.cc:1403] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.567s	user 0.031s	sys 0.077s
I20260709 13:17:32.351950 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.353600 16785 raft_consensus.cc:359] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.354317 16785 raft_consensus.cc:385] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.354830 16785 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.355558 16785 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.360571 16785 ts_tablet_manager.cc:1434] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:17:32.354179 16787 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:32.362061 16787 ts_tablet_manager.cc:1403] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.155s	user 0.005s	sys 0.011s
I20260709 13:17:32.362756 17171 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.363993 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:32.365283 16785 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
W20260709 13:17:32.366943 16708 leader_election.cc:343] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.367296 16708 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.368207 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.369097 17298 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.376832 17260 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.377367 17260 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:32.378190 17260 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.378451 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:32.366025 17171 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:32.385666 17171 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.386615 16588 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.397137 17301 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.397724 17301 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.413000 17171 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.424978 16787 raft_consensus.cc:359] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.425860 16787 raft_consensus.cc:385] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.426213 16787 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.427037 16787 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.430502 16786 tablet_bootstrap.cc:654] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.424379 17171 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.432935 17171 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.433969 17301 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.445217 16785 tablet_bootstrap.cc:654] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.448956 16787 ts_tablet_manager.cc:1434] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.001s
I20260709 13:17:32.450537 16787 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:32.454164 17301 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:32.457000 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:32.457765 16670 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:32.458954 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:32.467804 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:32.469707 16708 leader_election.cc:343] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.478416 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.477279 17162 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:32.487039 16708 leader_election.cc:343] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.478119 17301 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.496061 16493 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:32.492619 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.498584 16493 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:32.495738 17301 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.509568 17301 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.523362 17301 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.525447 17301 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.531896 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:32.532621 16668 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.534210 16786 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:32.534296 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:32.534942 16786 ts_tablet_manager.cc:1403] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.197s	user 0.020s	sys 0.001s
W20260709 13:17:32.537345 16708 leader_election.cc:343] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.537997 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.540544 17292 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:32.544450 16785 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:32.544957 16785 ts_tablet_manager.cc:1403] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.180s	user 0.008s	sys 0.011s
I20260709 13:17:32.552009 16786 raft_consensus.cc:359] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.552625 16786 raft_consensus.cc:385] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.552841 16786 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.562093 16668 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:17:32.553342 16786 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.564316 16785 raft_consensus.cc:359] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.565019 16785 raft_consensus.cc:385] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.565263 16785 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.565953 16785 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.570120 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:32.576072 16708 leader_election.cc:343] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.571306 16592 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.573061 17292 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.577839 16785 ts_tablet_manager.cc:1434] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.002s
I20260709 13:17:32.577855 16786 ts_tablet_manager.cc:1434] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.042s	user 0.003s	sys 0.004s
I20260709 13:17:32.581100 17292 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:32.581969 17292 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.582306 16785 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:32.584705 16787 tablet_bootstrap.cc:654] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.587757 16786 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Bootstrap starting.
I20260709 13:17:32.595906 17171 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.596650 17171 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.598496 17292 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.599191 17171 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.599083 17292 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.634684 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.644755 17292 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.605816 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:17:32.647553 16708 leader_election.cc:343] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.649715 16787 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:32.650713 16787 ts_tablet_manager.cc:1403] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.200s	user 0.007s	sys 0.006s
I20260709 13:17:32.652371 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.653191 16787 raft_consensus.cc:359] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.653844 16787 raft_consensus.cc:385] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.654117 16787 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.654794 16787 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.656080 16785 tablet_bootstrap.cc:654] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:32.656803 16708 leader_election.cc:343] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.656836 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:32.657341 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.657716 16670 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 0.
I20260709 13:17:32.658245 17223 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:17:32.659734 16708 leader_election.cc:343] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.660420 16786 tablet_bootstrap.cc:654] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.660392 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.661268 17223 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.661684 17223 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.662074 17223 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.668565 16787 ts_tablet_manager.cc:1434] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:17:32.669667 16787 tablet_bootstrap.cc:492] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Bootstrap starting.
I20260709 13:17:32.701910 17223 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.710475 16785 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:32.711130 16785 ts_tablet_manager.cc:1403] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.130s	user 0.011s	sys 0.002s
I20260709 13:17:32.713560 17166 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.714941 17166 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.718322 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:32.718971 17166 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:17:32.720063 16708 leader_election.cc:343] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.721123 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:32.721300 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.721660 16744 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:32.722749 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:32.723505 17295 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:32.723881 17295 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.723942 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:32.724298 17295 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.724665 16666 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.724833 16785 raft_consensus.cc:359] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.726660 16785 raft_consensus.cc:385] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.727233 16785 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.727856 16785 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:32.728400 16634 leader_election.cc:343] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.721408 17223 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:32.732244 16787 tablet_bootstrap.cc:654] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.734440 16493 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:32.741475 16786 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: No bootstrap required, opened a new log
I20260709 13:17:32.742033 16786 ts_tablet_manager.cc:1403] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Time spent bootstrapping tablet: real 0.155s	user 0.021s	sys 0.034s
I20260709 13:17:32.743305 16785 ts_tablet_manager.cc:1434] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.002s
I20260709 13:17:32.747088 16785 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:32.749969 16786 raft_consensus.cc:359] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.751497 16786 raft_consensus.cc:385] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.751868 16786 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.754587 16786 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.754591 17295 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.761025 16786 ts_tablet_manager.cc:1434] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 13:17:32.766008 17295 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:32.766850 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:32.766997 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:32.768990 16742 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.767652 16666 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
W20260709 13:17:32.772219 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.767678 17167 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:32.780216 16742 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:32.781632 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:17:32.782357 16634 leader_election.cc:343] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:32.789131 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:32.794097 17223 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.787279 17300 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:32.800972 17300 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:32.796193 17223 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:32.802371 16787 tablet_bootstrap.cc:492] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: No bootstrap required, opened a new log
I20260709 13:17:32.802300 17223 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.787357 17167 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.802922 16787 ts_tablet_manager.cc:1403] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Time spent bootstrapping tablet: real 0.133s	user 0.015s	sys 0.005s
I20260709 13:17:32.801942 17300 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.805305 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:32.806458 16592 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:17:32.807737 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:32.805464 17167 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:32.808441 17245 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:32.809051 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:32.810113 17245 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:32.810489 17245 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:32.812034 16785 tablet_bootstrap.cc:654] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:32.828595 16708 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:32.832274 17302 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:32.834695 17302 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:32.861773 16787 raft_consensus.cc:359] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.865660 16787 raft_consensus.cc:385] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.866104 16787 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.871884 16787 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.874037 16787 ts_tablet_manager.cc:1434] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Time spent starting tablet: real 0.070s	user 0.004s	sys 0.002s
I20260709 13:17:32.864670 17245 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.884404 16596 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:32.886281 16494 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:32.887153 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:32.887063 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:32.887856 16588 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:32.890674 17245 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:32.881060 16785 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:32.904414 16785 ts_tablet_manager.cc:1403] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.158s	user 0.010s	sys 0.002s
I20260709 13:17:32.908396 16785 raft_consensus.cc:359] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.909149 16785 raft_consensus.cc:385] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:32.909397 16785 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:32.910065 16785 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.912294 16785 ts_tablet_manager.cc:1434] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:32.929713 16588 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:17:32.933893 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:32.934798 17300 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:32.935402 17300 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:32.936340 17300 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.938012 16785 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:32.936033 16493 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:32.975968 16785 tablet_bootstrap.cc:654] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:32.996852 16785 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:32.997421 16785 ts_tablet_manager.cc:1403] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.000s
I20260709 13:17:33.000119 16785 raft_consensus.cc:359] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.000794 16785 raft_consensus.cc:385] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.001052 16785 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.001711 16785 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:32.994118 16493 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.035154 16785 ts_tablet_manager.cc:1434] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 13:17:33.036227 16785 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:33.045732 17300 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.046277 17300 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.048473 17300 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.050041 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:33.051853 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:17:33.052098 16634 leader_election.cc:343] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.052461 16744 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:33.053210 16785 tablet_bootstrap.cc:654] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.053566 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:33.054498 17300 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.054836 17300 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.055189 17300 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.059235 17292 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:17:33.059892 17292 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.061808 17300 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.062431 17292 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:33.064378 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:33.064525 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:33.065250 16596 raft_consensus.cc:2368] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:17:33.065426 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:33.066460 16708 leader_election.cc:400] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:17:33.066782 16708 leader_election.cc:403] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:33.066975 16634 leader_election.cc:343] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.068135 17223 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.074729 17300 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.075785 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:33.076386 16744 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:33.078795 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:33.085191 17223 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:17:33.087891 16744 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:33.089331 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:33.090122 17245 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:33.090629 17245 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:33.091454 17245 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.094077 16785 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:33.094631 16785 ts_tablet_manager.cc:1403] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.059s	user 0.019s	sys 0.000s
I20260709 13:17:33.099575 16785 raft_consensus.cc:359] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.101439 16785 raft_consensus.cc:385] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.101775 16785 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.102500 16785 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.105011 16785 ts_tablet_manager.cc:1434] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:17:33.107055 17162 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.108389 17162 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.110803 16785 tablet_bootstrap.cc:492] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:33.112257 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.110266 17162 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.112936 16744 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:33.113746 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:33.115234 16561 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:33.121487 16785 tablet_bootstrap.cc:654] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.126317 17304 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.139478 16493 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.144184 17304 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:33.145579 17162 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.146078 17162 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.149163 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:33.151186 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.152464 16785 tablet_bootstrap.cc:492] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:33.152653 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:33.152928 16785 ts_tablet_manager.cc:1403] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.006s
I20260709 13:17:33.155589 16785 raft_consensus.cc:359] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.156296 16785 raft_consensus.cc:385] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.156579 16785 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.156638 17162 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.157239 16785 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.159595 16785 ts_tablet_manager.cc:1434] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:17:33.161092 16785 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:33.164597 17245 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.165130 17245 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.169548 17300 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.169466 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:33.170102 17300 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.170770 16596 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:33.172225 17300 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.173326 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:33.174446 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:17:33.175179 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.175498 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.175767 16742 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
W20260709 13:17:33.175920 16634 leader_election.cc:343] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.173408 17245 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.176116 16741 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 0.
I20260709 13:17:33.176574 17245 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.176959 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:33.177785 17109 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:33.178125 17109 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.178443 17109 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.176972 17245 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:33.181043 17245 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.184587 17223 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.186802 17109 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.192104 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:33.192847 17304 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.193383 17304 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:17:33.193714 16634 leader_election.cc:343] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:33.198671 17109 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.200065 16785 tablet_bootstrap.cc:654] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.205679 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:33.206357 16744 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.205737 17245 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.157789 17162 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:33.211301 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:33.211935 16596 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:33.213163 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60586.2us
I20260709 13:17:33.219907 16596 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:33.221961 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:33.222787 17109 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:33.223308 17109 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:33.224149 17109 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.228770 16593 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.237943 17167 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.245074 17245 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.250417 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:33.251091 16741 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.269356 16596 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.284192 16741 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:33.290288 17304 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.297140 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:33.291561 16744 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:17:33.306806 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:17:33.312443 16542 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:17:33.316566 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:33.309088 17109 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:33.312820 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.320996 17109 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:33.324987 17109 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.331727 16785 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:33.359966 16741 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:33.365306 16785 ts_tablet_manager.cc:1403] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.204s	user 0.019s	sys 0.012s
I20260709 13:17:33.365824 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:33.372292 17293 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.379046 16785 raft_consensus.cc:359] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.379707 16785 raft_consensus.cc:385] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.379973 16785 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.389297 16785 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.398713 16785 ts_tablet_manager.cc:1434] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.003s
I20260709 13:17:33.395121 17162 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:33.418172 16742 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:33.420231 16785 tablet_bootstrap.cc:492] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:33.420039 17300 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.433629 17301 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.458719 16494 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.461015 16494 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 0 to 1, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.464977 16785 tablet_bootstrap.cc:654] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.466480 16593 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:33.471047 17245 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.502962 16785 tablet_bootstrap.cc:492] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:33.503454 16785 ts_tablet_manager.cc:1403] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.084s	user 0.012s	sys 0.000s
I20260709 13:17:33.506036 16785 raft_consensus.cc:359] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.506732 16785 raft_consensus.cc:385] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.507032 16785 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.507604 16785 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.524359 17293 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.527791 16785 ts_tablet_manager.cc:1434] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 13:17:33.532387 16785 tablet_bootstrap.cc:492] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:33.560571 17293 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.577785 16785 tablet_bootstrap.cc:654] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.615823 17344 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.620604 16785 tablet_bootstrap.cc:492] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:33.621134 16785 ts_tablet_manager.cc:1403] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.089s	user 0.012s	sys 0.015s
I20260709 13:17:33.623804 16785 raft_consensus.cc:359] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.624454 16785 raft_consensus.cc:385] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.624719 16785 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.625365 16785 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.627209 16785 ts_tablet_manager.cc:1434] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:17:33.628139 16785 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Bootstrap starting.
I20260709 13:17:33.645337 16785 tablet_bootstrap.cc:654] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:33.645159 17293 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.683575 17337 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.684101 17337 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.684278 16592 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.686087 17337 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.687570 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:33.688073 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.688763 16744 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 1: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:17:33.690291 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:33.692646 17162 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.704993 16785 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: No bootstrap required, opened a new log
I20260709 13:17:33.705521 16785 ts_tablet_manager.cc:1403] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.010s
I20260709 13:17:33.709309 16785 raft_consensus.cc:359] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.710000 16785 raft_consensus.cc:385] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:33.710285 16785 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Initialized, Role: FOLLOWER
I20260709 13:17:33.716100 17162 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:33.720907 16785 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.723332 16785 ts_tablet_manager.cc:1434] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:17:33.830102 16592 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:33.863031 17304 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:33.863509 17304 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:33.865545 17304 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:33.867132 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:17:33.867962 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:17:33.870209 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:17:33.870944 17304 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:33.881966 17293 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:33.923991 17344 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:34.063514 16592 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:35.219949 15476 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:35.241858 17363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:35.243562 17364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:35.278250 17368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:35.280372 15476 server_base.cc:1061] running on GCE node
I20260709 13:17:35.283401 15476 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:35.283711 15476 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:35.285764 15476 hybrid_clock.cc:648] HybridClock initialized: now 1783603055285739 us; error 0 us; skew 500 ppm
I20260709 13:17:35.286701 15476 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:35.311686 15476 webserver.cc:533] Webserver started at http://127.15.29.4:38463/ using document root <none> and password file <none>
I20260709 13:17:35.312484 15476 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:35.312777 15476 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:35.313242 15476 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:35.315303 15476 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-3-root/instance:
uuid: "ba2df6a668144022a223e504ae571f21"
format_stamp: "Formatted at 2026-07-09 13:17:35 on dist-test-slave-hz8m"
I20260709 13:17:35.329180 15476 fs_manager.cc:696] Time spent creating directory manager: real 0.013s	user 0.008s	sys 0.004s
I20260709 13:17:35.341354 17373 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:35.342454 15476 fs_manager.cc:730] Time spent opening block manager: real 0.010s	user 0.002s	sys 0.007s
I20260709 13:17:35.342911 15476 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-3-root
uuid: "ba2df6a668144022a223e504ae571f21"
format_stamp: "Formatted at 2026-07-09 13:17:35 on dist-test-slave-hz8m"
I20260709 13:17:35.343588 15476 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:35.493927 15476 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:35.512030 15476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:35.573536 15476 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:35.608824 15476 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:35.609262 15476 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:17:35.609743 15476 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:35.610106 15476 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
W20260709 13:17:36.564510 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84046.1us
I20260709 13:17:36.692296 15476 rpc_server.cc:307] RPC server started. Bound to: 127.15.29.4:41455
I20260709 13:17:36.692490 17446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.15.29.4:41455 every 8 connection(s)
I20260709 13:17:36.724957 15476 auto_leader_rebalancer-test.cc:384] add a tserver: ba2df6a668144022a223e504ae571f21
I20260709 13:17:36.743440 17447 heartbeater.cc:344] Connected to a master server at 127.15.29.62:37247
I20260709 13:17:36.744073 17447 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:36.749619 17447 heartbeater.cc:507] Master 127.15.29.62:37247 requested a full tablet report, sending...
I20260709 13:17:36.752214 16494 ts_manager.cc:194] Registered new tserver with Master: ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:17:36.754132 16494 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34978
I20260709 13:17:37.757965 17447 heartbeater.cc:499] Master 127.15.29.62:37247 was elected leader, sending a full tablet report...
W20260709 13:17:37.819831 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56471.6us
W20260709 13:17:37.958092 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49176.6us
I20260709 13:17:38.070960 16668 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:38.102793 16592 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.104671 17379 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:38.113305 16742 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.116767 17379 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.158981 17451 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:17:38.163898 16636 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c74caaa7fe045f9a0f6ff3b3df97602. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.166020 16742 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.178243 16593 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.192173 16494 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.201535 16662 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:17:38.226135 16636 consensus_peers.cc:597] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 374e8578b72347509cc3ea1e6f464f31. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.236666 16742 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.247905 16593 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.249648 17451 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.265133 17451 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:17:38.275605 17379 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.292826 16589 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.331933 16742 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.341081 16743 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:38.374171 16493 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.384859 16662 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.386817 16589 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.386823 17366 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:17:38.390852 17417 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:38.489017 16710 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 173ca11ec288406aa68fe01a80d48e30. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.494815 17417 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.522760 16668 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.526624 16741 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:38.541862 16493 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:17:38.553889 16710 consensus_peers.cc:597] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04d635b36e854e4f9ce8b973de8edafe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:38.565131 16668 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.565150 16592 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:38.574460 17476 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:38.577651 16592 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.585386 17478 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:38.600040 17479 ts_tablet_manager.cc:933] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:38.628901 17476 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.634449 17479 tablet_copy_client.cc:323] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:38.632331 16589 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.650684 16668 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.674193 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 7c74caaa7fe045f9a0f6ff3b3df97602 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:38.674834 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 7c74caaa7fe045f9a0f6ff3b3df97602 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-7c74caaa7fe045f9a0f6ff3b3df97602
I20260709 13:17:38.676904 16493 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:38.724247 16681 tablet_copy_source_session.cc:215] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:38.752461 17479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c74caaa7fe045f9a0f6ff3b3df97602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.817508 17485 ts_tablet_manager.cc:933] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:38.849181 17485 tablet_copy_client.cc:323] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:38.842623 17479 tablet_copy_client.cc:806] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:38.851150 17479 tablet_copy_client.cc:670] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:38.851230 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 374e8578b72347509cc3ea1e6f464f31 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:38.851881 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 374e8578b72347509cc3ea1e6f464f31 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-374e8578b72347509cc3ea1e6f464f31
I20260709 13:17:38.861338 17479 tablet_copy_client.cc:538] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:38.862007 16681 tablet_copy_source_session.cc:215] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:38.871482 17485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 374e8578b72347509cc3ea1e6f464f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:38.881407 17479 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:38.899962 17487 ts_tablet_manager.cc:933] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:38.905205 17485 tablet_copy_client.cc:806] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:38.908180 17485 tablet_copy_client.cc:670] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:38.912070 17485 tablet_copy_client.cc:538] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:38.921950 17487 tablet_copy_client.cc:323] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:38.935662 17485 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:38.963754 17491 ts_tablet_manager.cc:933] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:38.969727 17479 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:38.974140 17479 tablet_bootstrap.cc:492] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:38.974797 17479 ts_tablet_manager.cc:1403] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.014s
I20260709 13:17:38.978510 17479 raft_consensus.cc:359] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:38.979552 17479 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:38.981523 17479 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:38.990355 17491 tablet_copy_client.cc:323] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:38.998672 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 173ca11ec288406aa68fe01a80d48e30 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:38.999505 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 173ca11ec288406aa68fe01a80d48e30 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-173ca11ec288406aa68fe01a80d48e30
I20260709 13:17:39.003779 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 04d635b36e854e4f9ce8b973de8edafe from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:39.004455 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 04d635b36e854e4f9ce8b973de8edafe from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-04d635b36e854e4f9ce8b973de8edafe
I20260709 13:17:39.020365 16754 tablet_copy_source_session.cc:215] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:39.028836 17485 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:39.022315 16755 tablet_copy_source_session.cc:215] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:39.032655 17479 ts_tablet_manager.cc:1434] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.058s	user 0.009s	sys 0.007s
I20260709 13:17:39.032826 17487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 173ca11ec288406aa68fe01a80d48e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:39.042197 17491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04d635b36e854e4f9ce8b973de8edafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:39.048091 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-7c74caaa7fe045f9a0f6ff3b3df97602 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:39.048738 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-7c74caaa7fe045f9a0f6ff3b3df97602 on tablet 7c74caaa7fe045f9a0f6ff3b3df97602 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:39.057449 17420 raft_consensus.cc:1217] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:39.078951 17485 tablet_bootstrap.cc:492] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:39.079939 17485 ts_tablet_manager.cc:1403] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.014s
I20260709 13:17:39.099188 17491 tablet_copy_client.cc:806] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:39.088562 17485 raft_consensus.cc:359] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:39.100042 17485 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:39.100841 17485 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:39.103147 17485 ts_tablet_manager.cc:1434] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 13:17:39.106006 17487 tablet_copy_client.cc:806] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:39.106573 17487 tablet_copy_client.cc:670] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:39.100034 17491 tablet_copy_client.cc:670] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:39.114390 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-374e8578b72347509cc3ea1e6f464f31 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:39.123351 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-374e8578b72347509cc3ea1e6f464f31 on tablet 374e8578b72347509cc3ea1e6f464f31 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:39.132730 17491 tablet_copy_client.cc:538] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:39.134891 17487 tablet_copy_client.cc:538] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:39.152307 17491 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:39.160557 17487 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:39.267036 17491 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:39.267954 17491 tablet_bootstrap.cc:492] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:39.270046 17487 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:39.271844 17487 tablet_bootstrap.cc:492] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:39.272490 17487 ts_tablet_manager.cc:1403] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.008s
I20260709 13:17:39.275098 17491 ts_tablet_manager.cc:1403] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.123s	user 0.022s	sys 0.005s
I20260709 13:17:39.276634 17487 raft_consensus.cc:359] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:39.279454 17487 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:39.281900 17487 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:39.280773 17491 raft_consensus.cc:359] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:39.299974 17491 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:39.300889 17491 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:39.303272 17491 ts_tablet_manager.cc:1434] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:17:39.305110 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-04d635b36e854e4f9ce8b973de8edafe received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:39.305547 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-04d635b36e854e4f9ce8b973de8edafe on tablet 04d635b36e854e4f9ce8b973de8edafe with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:39.299491 17487 ts_tablet_manager.cc:1434] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:17:39.324689 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-173ca11ec288406aa68fe01a80d48e30 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:39.325162 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-173ca11ec288406aa68fe01a80d48e30 on tablet 173ca11ec288406aa68fe01a80d48e30 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:39.372993 17420 raft_consensus.cc:1217] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:39.492836 17420 raft_consensus.cc:1217] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:39.502311 17420 raft_consensus.cc:1217] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:39.517442 17498 raft_consensus.cc:1064] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:39.521515 17498 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:39.568074 16593 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.574880 16741 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.579697 17420 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.576872 17474 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:17:39.584467 17498 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:39.585964 17473 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.690372 17501 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.701237 16592 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.696981 16741 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.718375 17420 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.737789 16493 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:17:39.756434 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 68074.5us
I20260709 13:17:39.824508 16667 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:39.834096 17420 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.834661 16593 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:39.836690 17473 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:39.843490 17498 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:39.853765 17501 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.862553 16592 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.882423 17499 raft_consensus.cc:1064] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:39.884698 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7c74caaa7fe045f9a0f6ff3b3df97602 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:39.892769 16494 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.862187 17420 raft_consensus.cc:2955] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:39.903744 17499 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:39.932502 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet 7c74caaa7fe045f9a0f6ff3b3df97602 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:39.940598 17511 tablet_replica.cc:333] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:39.956281 17420 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.958046 17499 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.960578 16593 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.963897 16667 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:39.970294 17476 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:39.973417 17478 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:39.994634 17473 raft_consensus.cc:1064] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:39.997354 17473 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:40.011241 17478 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.013528 17419 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:40.015703 16742 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:40.015815 16589 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.017922 17498 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:40.019853 17498 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:40.026106 16592 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:40.030668 17451 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:40.076308 17511 raft_consensus.cc:2243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:40.077220 17511 raft_consensus.cc:2272] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:40.076416 16491 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.085026 17511 ts_tablet_manager.cc:1916] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:40.088846 17505 raft_consensus.cc:1064] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:40.150390 16667 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.152436 17501 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.159602 17419 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.152802 17505 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:40.169312 16736 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.222482 16592 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:40.230382 16662 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:40.236330 17416 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:40.235826 17420 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.240186 17505 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:17:40.245700 17505 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:40.249745 17499 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:40.275386 16589 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.286047 16494 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:40.306131 17476 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.307945 17420 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.307946 17511 ts_tablet_manager.cc:1929] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:40.311810 17511 log.cc:1199] T 7c74caaa7fe045f9a0f6ff3b3df97602 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/7c74caaa7fe045f9a0f6ff3b3df97602
I20260709 13:17:40.342375 16662 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.354368 16668 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:40.362687 16592 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.368057 16493 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.374675 17416 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:40.377617 17473 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:40.385059 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet 7c74caaa7fe045f9a0f6ff3b3df97602 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:40.392835 16736 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:40.402361 17451 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.413949 16589 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:40.414670 17416 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:40.417207 17476 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:40.418349 17521 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:17:40.418047 17416 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.430552 17521 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.431720 17420 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.449577 16736 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:40.465306 17501 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:40.478776 16494 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.482786 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 173ca11ec288406aa68fe01a80d48e30 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:40.491608 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 374e8578b72347509cc3ea1e6f464f31 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:40.496240 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet 173ca11ec288406aa68fe01a80d48e30 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:40.499758 16491 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:40.494099 16592 raft_consensus.cc:2955] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.518033 16745 raft_consensus.cc:2955] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.522346 17529 tablet_replica.cc:333] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:40.531651 17529 raft_consensus.cc:2243] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:40.532325 17529 raft_consensus.cc:2272] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:40.539289 17529 ts_tablet_manager.cc:1916] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:40.542400 16736 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:40.545749 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 374e8578b72347509cc3ea1e6f464f31 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:40.555338 17416 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:40.556560 16662 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:40.556860 17521 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:40.559996 17521 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:40.565331 17478 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.580324 17530 tablet_replica.cc:333] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:40.581554 16668 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.584125 17416 raft_consensus.cc:2955] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.590334 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 04d635b36e854e4f9ce8b973de8edafe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:40.602828 17529 ts_tablet_manager.cc:1929] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:40.603380 17529 log.cc:1199] T 173ca11ec288406aa68fe01a80d48e30 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/173ca11ec288406aa68fe01a80d48e30
I20260709 13:17:40.604869 16494 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:40.607486 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet 173ca11ec288406aa68fe01a80d48e30 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:40.641593 17530 raft_consensus.cc:2243] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:40.641871 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 04d635b36e854e4f9ce8b973de8edafe with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:40.660369 17530 raft_consensus.cc:2272] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:40.677510 17530 ts_tablet_manager.cc:1916] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:40.706131 17530 ts_tablet_manager.cc:1929] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:40.706609 17530 log.cc:1199] T 374e8578b72347509cc3ea1e6f464f31 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/374e8578b72347509cc3ea1e6f464f31
I20260709 13:17:40.735157 17530 tablet_replica.cc:333] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:40.735914 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 374e8578b72347509cc3ea1e6f464f31 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:40.736326 17530 raft_consensus.cc:2243] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:40.737083 17530 raft_consensus.cc:2272] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:40.745206 17530 ts_tablet_manager.cc:1916] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:40.780498 17530 ts_tablet_manager.cc:1929] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:40.780937 17530 log.cc:1199] T 04d635b36e854e4f9ce8b973de8edafe P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/04d635b36e854e4f9ce8b973de8edafe
I20260709 13:17:40.786159 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 04d635b36e854e4f9ce8b973de8edafe (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:41.815287 16589 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:41.864960 16662 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.867442 16745 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:41.866524 17517 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:17:41.869153 17535 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:41.876628 16561 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9b74fd3415d14ae3aeb38c349eca0494. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:41.965771 17517 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:41.974695 16745 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:41.987977 16494 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:41.997292 16739 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:17:42.015023 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 69846us
W20260709 13:17:42.056113 16710 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 92129e39cb38498ea9f0addc0a33f247. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:42.061654 16667 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.098312 16662 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:42.101574 16592 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:42.141773 17540 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.149989 17540 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.180660 17545 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.183686 16592 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.209280 16667 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.223589 16493 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.231076 16662 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.266340 17552 ts_tablet_manager.cc:933] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:42.292701 17552 tablet_copy_client.cc:323] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:17:42.326483 16739 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:42.329025 16592 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:42.331063 17531 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:42.332726 16636 consensus_peers.cc:597] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 04867b8a53e645f0b58c55ecec0dac8c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:42.332803 17528 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.343966 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet 9b74fd3415d14ae3aeb38c349eca0494 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:17:42.344657 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet 9b74fd3415d14ae3aeb38c349eca0494 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-9b74fd3415d14ae3aeb38c349eca0494
I20260709 13:17:42.352757 17557 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.357740 16606 tablet_copy_source_session.cc:215] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:42.367379 16589 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.371960 16592 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.375922 17552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b74fd3415d14ae3aeb38c349eca0494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.382669 16493 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.382426 16736 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.425401 17552 tablet_copy_client.cc:806] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:42.425987 17552 tablet_copy_client.cc:670] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:42.432260 16745 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:42.434228 17439 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:42.435195 16561 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee99829f455541f09dac21015b00282e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:42.435443 16662 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:42.436205 17552 tablet_copy_client.cc:538] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:42.437078 17439 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.443742 17517 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.445462 16745 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.461012 16667 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.465649 16493 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:42.476538 17552 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:42.537868 17563 ts_tablet_manager.cc:933] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:42.580646 17563 tablet_copy_client.cc:323] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:42.595381 17552 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:42.596378 17552 tablet_bootstrap.cc:492] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:42.597067 17552 ts_tablet_manager.cc:1403] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.121s	user 0.022s	sys 0.029s
I20260709 13:17:42.597951 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 92129e39cb38498ea9f0addc0a33f247 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:42.598523 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 92129e39cb38498ea9f0addc0a33f247 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-92129e39cb38498ea9f0addc0a33f247
I20260709 13:17:42.600175 17552 raft_consensus.cc:359] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.603197 17552 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:42.604437 17552 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.636238 16755 tablet_copy_source_session.cc:215] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:42.641083 17563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92129e39cb38498ea9f0addc0a33f247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.650378 17552 ts_tablet_manager.cc:1434] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.053s	user 0.023s	sys 0.020s
I20260709 13:17:42.653442 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-9b74fd3415d14ae3aeb38c349eca0494 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:17:42.653949 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-9b74fd3415d14ae3aeb38c349eca0494 on tablet 9b74fd3415d14ae3aeb38c349eca0494 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:42.665432 17563 tablet_copy_client.cc:806] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:42.666059 17563 tablet_copy_client.cc:670] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:42.675292 17563 tablet_copy_client.cc:538] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:42.694754 17563 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:42.720760 17563 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:42.721597 17563 tablet_bootstrap.cc:492] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:42.722290 17563 ts_tablet_manager.cc:1403] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.028s	user 0.025s	sys 0.001s
I20260709 13:17:42.724792 17563 raft_consensus.cc:359] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.725409 17563 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:42.725906 17563 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.727766 17563 ts_tablet_manager.cc:1434] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.006s
I20260709 13:17:42.729575 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-92129e39cb38498ea9f0addc0a33f247 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:42.729985 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-92129e39cb38498ea9f0addc0a33f247 on tablet 92129e39cb38498ea9f0addc0a33f247 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:42.814402 17563 ts_tablet_manager.cc:933] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:42.816232 17563 tablet_copy_client.cc:323] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:42.817817 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 04867b8a53e645f0b58c55ecec0dac8c from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:42.818301 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 04867b8a53e645f0b58c55ecec0dac8c from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-04867b8a53e645f0b58c55ecec0dac8c
I20260709 13:17:42.827277 16681 tablet_copy_source_session.cc:215] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:42.832610 17563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04867b8a53e645f0b58c55ecec0dac8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.851528 17563 tablet_copy_client.cc:806] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:42.852432 17563 tablet_copy_client.cc:670] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:42.855840 17416 raft_consensus.cc:1217] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:42.868930 17552 ts_tablet_manager.cc:933] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:42.873099 17563 tablet_copy_client.cc:538] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:42.885996 17563 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:42.887293 17552 tablet_copy_client.cc:323] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:17:42.889582 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet ee99829f455541f09dac21015b00282e from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:17:42.890139 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet ee99829f455541f09dac21015b00282e from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-ee99829f455541f09dac21015b00282e
I20260709 13:17:42.898309 16606 tablet_copy_source_session.cc:215] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:42.901412 17552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee99829f455541f09dac21015b00282e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:42.910504 17563 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:42.911422 17563 tablet_bootstrap.cc:492] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:42.912134 17563 ts_tablet_manager.cc:1403] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.004s
I20260709 13:17:42.915365 17563 raft_consensus.cc:359] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.916308 17563 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:42.917140 17552 tablet_copy_client.cc:806] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:42.917011 17563 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.917719 17552 tablet_copy_client.cc:670] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:42.919886 17563 ts_tablet_manager.cc:1434] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.005s
I20260709 13:17:42.928501 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-04867b8a53e645f0b58c55ecec0dac8c received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:42.928972 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-04867b8a53e645f0b58c55ecec0dac8c on tablet 04867b8a53e645f0b58c55ecec0dac8c with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:42.934384 17552 tablet_copy_client.cc:538] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:42.945153 17552 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:42.965830 17552 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:42.966740 17552 tablet_bootstrap.cc:492] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:42.966853 17416 raft_consensus.cc:1217] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:42.967721 17552 ts_tablet_manager.cc:1403] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.006s
I20260709 13:17:42.971304 17552 raft_consensus.cc:359] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.972108 17552 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:42.972731 17552 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:42.976162 17552 ts_tablet_manager.cc:1434] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:42.984407 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-ee99829f455541f09dac21015b00282e received from {username='slave'} at 127.0.0.1:47022
I20260709 13:17:42.984932 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-ee99829f455541f09dac21015b00282e on tablet ee99829f455541f09dac21015b00282e with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:43.359495 17544 raft_consensus.cc:1064] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:43.362639 17544 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:43.369954 17416 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.370627 16588 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.371587 16668 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.371630 17544 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:43.372650 17417 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.373518 17540 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.390532 17545 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.396186 16588 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.404131 17416 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.400187 16667 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.430658 16492 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:43.452632 17577 raft_consensus.cc:1064] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:43.456215 17577 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:43.451203 17416 raft_consensus.cc:1217] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:43.471582 17416 raft_consensus.cc:1217] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:43.533165 16736 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:43.549988 17420 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.551774 16588 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.553123 17540 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.559785 17540 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.564891 17417 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.570947 17420 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.574589 16588 raft_consensus.cc:2955] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.577759 17416 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.578290 16745 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.579025 16667 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.580209 17578 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.581147 17581 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.589237 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 92129e39cb38498ea9f0addc0a33f247 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:43.592469 16493 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.583636 17577 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:17:43.608421 17578 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.611465 17416 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.622684 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet 92129e39cb38498ea9f0addc0a33f247 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:43.648275 16492 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.653010 16745 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.666437 16493 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 92129e39cb38498ea9f0addc0a33f247 with cas_config_opid_index 3: aborting the task: latest config opid_index 4; task opid_index 3
I20260709 13:17:43.666538 16667 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.678725 16589 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:43.692124 17416 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.692731 16668 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:43.694312 17439 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.696947 17578 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:43.702526 17577 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.706763 17590 tablet_replica.cc:333] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:43.709939 17416 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.714366 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9b74fd3415d14ae3aeb38c349eca0494 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:43.715205 16667 raft_consensus.cc:2955] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.707913 17590 raft_consensus.cc:2243] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:43.723555 16492 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:43.731019 17590 raft_consensus.cc:2272] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:43.735216 17590 ts_tablet_manager.cc:1916] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:43.741638 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet 9b74fd3415d14ae3aeb38c349eca0494 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:43.743373 17591 tablet_replica.cc:333] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:43.744776 17591 raft_consensus.cc:2243] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:43.745560 17591 raft_consensus.cc:2272] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:43.749411 17591 ts_tablet_manager.cc:1916] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:43.750852 17590 ts_tablet_manager.cc:1929] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:43.751328 17590 log.cc:1199] T 92129e39cb38498ea9f0addc0a33f247 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/92129e39cb38498ea9f0addc0a33f247
I20260709 13:17:43.753170 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet 92129e39cb38498ea9f0addc0a33f247 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:43.765468 17591 ts_tablet_manager.cc:1929] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:43.765970 17591 log.cc:1199] T 9b74fd3415d14ae3aeb38c349eca0494 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/9b74fd3415d14ae3aeb38c349eca0494
I20260709 13:17:43.767632 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet 9b74fd3415d14ae3aeb38c349eca0494 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:43.884775 17577 raft_consensus.cc:1064] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:43.887511 17577 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:43.903113 16745 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.903113 17416 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.903352 16668 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:43.904874 17581 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.905694 17577 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.906424 17578 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:43.929453 17581 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.933427 17416 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.936758 16667 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.958101 16491 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:43.973471 16745 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.001075 16589 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:44.011370 17416 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:44.012871 17578 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:44.014036 16662 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:44.018054 17581 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:44.021466 17577 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.032794 17416 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.035660 16667 raft_consensus.cc:2955] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.039005 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ee99829f455541f09dac21015b00282e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:44.045691 17528 raft_consensus.cc:1064] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:44.047988 17528 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:44.056192 16491 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:44.063244 17416 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:44.064930 16588 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:44.066658 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet ee99829f455541f09dac21015b00282e with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:44.070096 17591 tablet_replica.cc:333] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:44.071305 17591 raft_consensus.cc:2243] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:44.072058 17591 raft_consensus.cc:2272] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:44.065919 16745 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:44.066834 17556 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:17:44.078959 17557 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:44.082703 17591 ts_tablet_manager.cc:1916] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:44.099936 17556 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:44.126411 17591 ts_tablet_manager.cc:1929] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:44.128121 17591 log.cc:1199] T ee99829f455541f09dac21015b00282e P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/ee99829f455541f09dac21015b00282e
I20260709 13:17:44.128171 17416 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.128525 16736 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.130786 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet ee99829f455541f09dac21015b00282e (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:44.126262 17586 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.146476 16492 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.152918 16589 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.170444 16667 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:44.177835 17416 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:44.179123 17598 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:44.181005 16736 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:44.182960 17598 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:44.186013 17598 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.188087 17420 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.195842 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 04867b8a53e645f0b58c55ecec0dac8c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:44.197095 16736 raft_consensus.cc:2955] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:44.201982 16493 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:44.211985 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 04867b8a53e645f0b58c55ecec0dac8c with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:44.215588 17602 tablet_replica.cc:333] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:44.216506 17602 raft_consensus.cc:2243] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:44.217109 17602 raft_consensus.cc:2272] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:44.220006 17602 ts_tablet_manager.cc:1916] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:44.234535 17602 ts_tablet_manager.cc:1929] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:44.234848 17602 log.cc:1199] T 04867b8a53e645f0b58c55ecec0dac8c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/04867b8a53e645f0b58c55ecec0dac8c
I20260709 13:17:44.236161 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 04867b8a53e645f0b58c55ecec0dac8c (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:45.276664 16745 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:17:45.283722 16710 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc977d4c44af4e80b58e4e0864085d9a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.284471 16662 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:45.285590 17417 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.285935 16589 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:45.287235 17417 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.294806 17605 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.296427 16662 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.298225 16592 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.311271 16492 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.328104 16667 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.341254 16745 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:45.341573 16636 consensus_peers.cc:597] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a931fb289f824ff28e993f8590e64b69. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.342368 17586 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.342751 16592 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:45.344094 17556 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.355144 17586 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.362537 16739 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.372713 16592 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.384094 16589 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.386981 16493 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:17:45.415603 16561 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3249701d37834e80bbe408e24ced7d54. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.416960 16745 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:45.418006 16667 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:45.418439 17616 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.422541 17616 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:17:45.429953 17616 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.431360 16745 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.439527 16667 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.449092 16662 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.450132 16493 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.467175 16592 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:45.469147 17586 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:45.478698 16745 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:45.481125 16636 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bab84fe76da941dc98416ce3f8c6f35b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:45.481356 17556 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:45.546167 16745 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.542400 17556 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.612676 16589 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:45.618498 16493 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:17:45.650091 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60689.5us
I20260709 13:17:45.705029 17629 ts_tablet_manager.cc:933] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:45.712083 17629 tablet_copy_client.cc:323] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:45.715103 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet cc977d4c44af4e80b58e4e0864085d9a from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:45.715878 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet cc977d4c44af4e80b58e4e0864085d9a from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-cc977d4c44af4e80b58e4e0864085d9a
I20260709 13:17:45.732671 16755 tablet_copy_source_session.cc:215] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.736169 17629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc977d4c44af4e80b58e4e0864085d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.752009 17629 tablet_copy_client.cc:806] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:45.752712 17629 tablet_copy_client.cc:670] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:45.761673 17629 tablet_copy_client.cc:538] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:45.772325 17629 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:45.796912 17629 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:45.797940 17629 tablet_bootstrap.cc:492] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:45.798659 17629 ts_tablet_manager.cc:1403] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.004s
I20260709 13:17:45.801966 17629 raft_consensus.cc:359] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.802815 17629 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:45.803565 17629 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.807179 17629 ts_tablet_manager.cc:1434] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.008s
I20260709 13:17:45.808849 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-cc977d4c44af4e80b58e4e0864085d9a received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:45.809281 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-cc977d4c44af4e80b58e4e0864085d9a on tablet cc977d4c44af4e80b58e4e0864085d9a with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:45.851929 17629 ts_tablet_manager.cc:933] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:45.853837 17629 tablet_copy_client.cc:323] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:45.856302 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet a931fb289f824ff28e993f8590e64b69 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:45.856936 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet a931fb289f824ff28e993f8590e64b69 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-a931fb289f824ff28e993f8590e64b69
I20260709 13:17:45.867151 16681 tablet_copy_source_session.cc:215] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.869925 17629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a931fb289f824ff28e993f8590e64b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.885334 17629 tablet_copy_client.cc:806] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:45.886327 17629 tablet_copy_client.cc:670] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:45.891839 17629 tablet_copy_client.cc:538] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:45.899890 17629 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:45.921145 17629 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:45.922132 17629 tablet_bootstrap.cc:492] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:45.922828 17629 ts_tablet_manager.cc:1403] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.009s
I20260709 13:17:45.925896 17629 raft_consensus.cc:359] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.926779 17629 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:45.927464 17629 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:45.929986 17629 ts_tablet_manager.cc:1434] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:17:45.931840 17635 ts_tablet_manager.cc:933] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:45.932005 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-a931fb289f824ff28e993f8590e64b69 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:45.932533 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-a931fb289f824ff28e993f8590e64b69 on tablet a931fb289f824ff28e993f8590e64b69 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:45.942278 17635 tablet_copy_client.cc:323] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:45.944285 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet bab84fe76da941dc98416ce3f8c6f35b from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:45.944859 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet bab84fe76da941dc98416ce3f8c6f35b from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-bab84fe76da941dc98416ce3f8c6f35b
I20260709 13:17:45.955152 16681 tablet_copy_source_session.cc:215] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:45.958642 17635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bab84fe76da941dc98416ce3f8c6f35b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.977967 17635 tablet_copy_client.cc:806] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:45.978518 17635 tablet_copy_client.cc:670] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:45.981966 17635 tablet_copy_client.cc:538] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:45.989969 17635 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:46.008978 17635 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:46.009642 17635 tablet_bootstrap.cc:492] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:46.010277 17635 ts_tablet_manager.cc:1403] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 13:17:46.012884 17635 raft_consensus.cc:359] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:46.013550 17635 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:46.014041 17635 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:46.016258 17635 ts_tablet_manager.cc:1434] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:17:46.019076 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-bab84fe76da941dc98416ce3f8c6f35b received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:46.019495 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-bab84fe76da941dc98416ce3f8c6f35b on tablet bab84fe76da941dc98416ce3f8c6f35b with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:46.035704 17635 ts_tablet_manager.cc:933] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:46.037531 17635 tablet_copy_client.cc:323] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:17:46.039810 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet 3249701d37834e80bbe408e24ced7d54 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:17:46.040283 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet 3249701d37834e80bbe408e24ced7d54 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-3249701d37834e80bbe408e24ced7d54
I20260709 13:17:46.046356 16606 tablet_copy_source_session.cc:215] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:46.049386 17635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3249701d37834e80bbe408e24ced7d54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:46.066630 17635 tablet_copy_client.cc:806] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:46.067234 17635 tablet_copy_client.cc:670] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:46.070849 17635 tablet_copy_client.cc:538] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:46.080411 17635 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:46.102171 17635 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:46.103119 17635 tablet_bootstrap.cc:492] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:46.103848 17635 ts_tablet_manager.cc:1403] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.000s
I20260709 13:17:46.107242 17635 raft_consensus.cc:359] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:46.108083 17635 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:46.108706 17635 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:46.118109 17635 ts_tablet_manager.cc:1434] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.003s
I20260709 13:17:46.123160 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-3249701d37834e80bbe408e24ced7d54 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:17:46.123636 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-3249701d37834e80bbe408e24ced7d54 on tablet 3249701d37834e80bbe408e24ced7d54 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:46.133635 17420 raft_consensus.cc:1217] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:46.447216 17420 raft_consensus.cc:1217] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:46.482350 17420 raft_consensus.cc:1217] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:46.492323 17416 raft_consensus.cc:1217] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:46.512750 17640 raft_consensus.cc:1064] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:46.515381 17640 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:46.522372 17420 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.523488 16662 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.524289 16589 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:46.523809 17640 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:46.525677 17625 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.528873 17625 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:46.545948 17625 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.547858 17420 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.552054 16592 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.561453 16493 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.568269 16662 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.591113 16745 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:46.599385 16592 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:46.599385 17420 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:46.605300 17625 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:46.607751 17640 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:46.612576 17545 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.622213 17420 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.624370 16589 raft_consensus.cc:2955] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:46.649194 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cc977d4c44af4e80b58e4e0864085d9a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:46.658986 16492 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:46.671352 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet cc977d4c44af4e80b58e4e0864085d9a with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:46.685689 17652 tablet_replica.cc:333] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:46.686719 17652 raft_consensus.cc:2243] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:46.687563 17652 raft_consensus.cc:2272] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:46.691258 17652 ts_tablet_manager.cc:1916] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:46.703761 17652 ts_tablet_manager.cc:1929] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:46.704146 17652 log.cc:1199] T cc977d4c44af4e80b58e4e0864085d9a P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/cc977d4c44af4e80b58e4e0864085d9a
I20260709 13:17:46.705569 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet cc977d4c44af4e80b58e4e0864085d9a (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:47.033560 17628 raft_consensus.cc:1064] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:47.036022 17628 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:47.042518 17420 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.042846 16739 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.044453 16589 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.044478 17628 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.046373 17556 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.049628 17598 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:17:47.074484 17556 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.084615 17420 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.091514 16739 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.098204 16491 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.116118 17616 raft_consensus.cc:1064] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:47.118825 17616 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:47.133867 17598 raft_consensus.cc:1064] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:47.136337 17598 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:47.141414 16589 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.153118 16745 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.155807 17420 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.156545 17641 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.168098 16668 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:47.180891 16745 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:47.182838 17641 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.183075 17654 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:47.196208 17654 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.198743 17416 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.199733 16739 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.201017 17660 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.202232 17628 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.202551 16736 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.243012 17419 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:47.248142 17657 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:17:47.251227 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a931fb289f824ff28e993f8590e64b69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:17:47.256397 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 76999.6us
I20260709 13:17:47.259989 16668 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.257292 17419 raft_consensus.cc:2955] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.266346 16493 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.276859 17663 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:47.280819 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet a931fb289f824ff28e993f8590e64b69 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:47.282181 17641 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.290979 17420 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.298146 17671 tablet_replica.cc:333] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:47.300424 16589 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:47.304687 17671 raft_consensus.cc:2243] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.305470 17671 raft_consensus.cc:2272] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.314838 17671 ts_tablet_manager.cc:1916] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:47.318624 17660 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:47.339993 17657 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.350250 17420 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.350137 16662 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.356058 16736 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.378449 16492 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:47.386397 16736 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.379997 16589 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.391148 17671 ts_tablet_manager.cc:1929] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:47.391613 17671 log.cc:1199] T a931fb289f824ff28e993f8590e64b69 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/a931fb289f824ff28e993f8590e64b69
I20260709 13:17:47.377954 16491 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:47.393183 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet a931fb289f824ff28e993f8590e64b69 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:47.428860 16588 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:47.433387 16668 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:47.439142 17420 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:47.440898 17616 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:47.441326 17420 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:47.442242 16592 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:47.443674 16736 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:47.443864 17661 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:47.446024 17598 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:47.448374 17663 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:47.453418 17664 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.453644 17598 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.459403 16736 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.461032 17420 raft_consensus.cc:2955] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.468241 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3249701d37834e80bbe408e24ced7d54 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:47.469189 16592 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.475495 16492 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.485042 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bab84fe76da941dc98416ce3f8c6f35b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:47.484280 17415 raft_consensus.cc:2955] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:47.495630 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet 3249701d37834e80bbe408e24ced7d54 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:47.490537 16493 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:47.510046 17675 tablet_replica.cc:333] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:47.514829 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet bab84fe76da941dc98416ce3f8c6f35b with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:47.528069 17676 tablet_replica.cc:333] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:47.528524 17675 raft_consensus.cc:2243] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.529201 17676 raft_consensus.cc:2243] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:47.529420 17675 raft_consensus.cc:2272] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.530004 17676 raft_consensus.cc:2272] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:47.532996 17675 ts_tablet_manager.cc:1916] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:47.533562 17676 ts_tablet_manager.cc:1916] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:47.546057 17675 ts_tablet_manager.cc:1929] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:47.546595 17675 log.cc:1199] T 3249701d37834e80bbe408e24ced7d54 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/3249701d37834e80bbe408e24ced7d54
I20260709 13:17:47.548651 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet 3249701d37834e80bbe408e24ced7d54 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:47.548785 17676 ts_tablet_manager.cc:1929] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:47.549341 17676 log.cc:1199] T bab84fe76da941dc98416ce3f8c6f35b P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/bab84fe76da941dc98416ce3f8c6f35b
I20260709 13:17:47.550962 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet bab84fe76da941dc98416ce3f8c6f35b (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:48.595309 16592 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:17:48.603299 16561 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 446649301a1c45808dccfebfadfed54c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:48.604025 16743 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:48.605145 16662 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:48.606007 17678 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.606981 17663 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.617623 17678 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.619204 16662 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.625142 16743 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.633931 16736 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:48.636018 16494 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.655138 16662 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:48.656129 16710 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79e87765622246cc907a538b35c52c64. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:48.656149 16592 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:48.656890 17650 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.658080 17651 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.677860 17651 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.683519 16588 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.687980 16668 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.694486 16736 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:48.702065 16491 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:48.715281 16710 consensus_peers.cc:597] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dc3ee5aa5844e5e9c7e3f566e36a561. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:48.719625 16668 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:48.721378 17651 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.729921 16592 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:48.732132 17686 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.734998 17651 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.736099 16662 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.750371 16668 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:48.746915 16592 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.756855 16494 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.762264 16736 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:48.763523 16636 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f919f8e7484c4f359aedc6ac05b45076. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:48.764765 17660 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.775841 16592 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:48.777375 17598 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:48.797081 17598 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.804507 16736 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.806061 16592 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:48.825027 16493 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:49.023833 17700 ts_tablet_manager.cc:933] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:49.025786 17700 tablet_copy_client.cc:323] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:17:49.027730 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet 446649301a1c45808dccfebfadfed54c from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:17:49.028256 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet 446649301a1c45808dccfebfadfed54c from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-446649301a1c45808dccfebfadfed54c
I20260709 13:17:49.033957 16606 tablet_copy_source_session.cc:215] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:49.036849 17700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 446649301a1c45808dccfebfadfed54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:49.052564 17700 tablet_copy_client.cc:806] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:49.053201 17700 tablet_copy_client.cc:670] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:49.057926 17700 tablet_copy_client.cc:538] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:49.067301 17700 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:49.087397 17700 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:49.088248 17700 tablet_bootstrap.cc:492] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:49.088881 17700 ts_tablet_manager.cc:1403] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:17:49.091658 17700 raft_consensus.cc:359] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.092444 17700 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:49.093060 17700 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.095832 17700 ts_tablet_manager.cc:1434] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:49.097534 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-446649301a1c45808dccfebfadfed54c received from {username='slave'} at 127.0.0.1:47022
I20260709 13:17:49.097947 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-446649301a1c45808dccfebfadfed54c on tablet 446649301a1c45808dccfebfadfed54c with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:49.179327 17700 ts_tablet_manager.cc:933] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:49.183550 17700 tablet_copy_client.cc:323] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:49.185081 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 79e87765622246cc907a538b35c52c64 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:49.185642 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 79e87765622246cc907a538b35c52c64 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-79e87765622246cc907a538b35c52c64
I20260709 13:17:49.193234 16755 tablet_copy_source_session.cc:215] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:49.197137 17700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79e87765622246cc907a538b35c52c64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:49.212884 17700 tablet_copy_client.cc:806] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:49.213483 17700 tablet_copy_client.cc:670] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:49.217423 17700 tablet_copy_client.cc:538] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:49.224110 17700 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:49.242466 17700 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:49.243362 17700 tablet_bootstrap.cc:492] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:49.243978 17700 ts_tablet_manager.cc:1403] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 13:17:49.246798 17700 raft_consensus.cc:359] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.247691 17700 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:49.248442 17700 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.251377 17700 ts_tablet_manager.cc:1434] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:49.253396 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-79e87765622246cc907a538b35c52c64 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:49.253901 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-79e87765622246cc907a538b35c52c64 on tablet 79e87765622246cc907a538b35c52c64 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:49.263065 17700 ts_tablet_manager.cc:933] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:49.265127 17700 tablet_copy_client.cc:323] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:49.266608 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet f919f8e7484c4f359aedc6ac05b45076 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:49.267194 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet f919f8e7484c4f359aedc6ac05b45076 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-f919f8e7484c4f359aedc6ac05b45076
I20260709 13:17:49.274801 16681 tablet_copy_source_session.cc:215] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:49.278307 17700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f919f8e7484c4f359aedc6ac05b45076. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:49.291379 17700 tablet_copy_client.cc:806] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:49.291873 17700 tablet_copy_client.cc:670] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:49.300421 17700 tablet_copy_client.cc:538] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:49.311679 17700 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:49.328135 17700 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:49.329011 17700 tablet_bootstrap.cc:492] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:49.329625 17700 ts_tablet_manager.cc:1403] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 13:17:49.332234 17700 raft_consensus.cc:359] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.333035 17700 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:49.333627 17700 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.335899 17700 ts_tablet_manager.cc:1434] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:49.344393 17708 ts_tablet_manager.cc:933] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:49.350417 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-f919f8e7484c4f359aedc6ac05b45076 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:49.351001 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-f919f8e7484c4f359aedc6ac05b45076 on tablet f919f8e7484c4f359aedc6ac05b45076 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:49.351545 17708 tablet_copy_client.cc:323] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:49.354293 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:49.354810 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-2dc3ee5aa5844e5e9c7e3f566e36a561
I20260709 13:17:49.361737 16755 tablet_copy_source_session.cc:215] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:49.364719 17708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:49.381260 17708 tablet_copy_client.cc:806] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:49.381819 17708 tablet_copy_client.cc:670] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:49.385587 17708 tablet_copy_client.cc:538] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:49.393653 17708 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:49.428938 17708 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:49.430070 17708 tablet_bootstrap.cc:492] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:49.430918 17708 ts_tablet_manager.cc:1403] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.000s
I20260709 13:17:49.437757 17708 raft_consensus.cc:359] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.438653 17708 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:49.439388 17708 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:49.443584 17708 ts_tablet_manager.cc:1434] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 13:17:49.448295 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-2dc3ee5aa5844e5e9c7e3f566e36a561 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:49.448741 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-2dc3ee5aa5844e5e9c7e3f566e36a561 on tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:49.559408 17415 raft_consensus.cc:1217] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:49.747860 17415 raft_consensus.cc:1217] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:49.793586 17415 raft_consensus.cc:1217] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:49.855160 17415 raft_consensus.cc:1217] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:50.132979 17710 raft_consensus.cc:1064] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:50.135558 17710 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.142114 17415 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.143594 16743 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.143865 16668 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.143718 17715 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:50.145494 17678 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.146258 17710 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.161124 17678 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.179184 16667 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.179538 17415 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.205780 16743 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.208526 16491 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.264722 17651 raft_consensus.cc:1064] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:50.268059 17660 raft_consensus.cc:1064] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:50.269021 17651 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.272285 17660 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.275658 17686 raft_consensus.cc:1064] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:50.282138 17686 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.297926 17415 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.300170 17650 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.307175 16588 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.309902 17650 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.319351 16668 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.319774 17416 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.321233 17650 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.339857 17420 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.340163 16662 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.341791 16588 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.342510 17651 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.342840 16589 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.344225 17651 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.350371 17697 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:50.352154 17650 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.360265 17415 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.364019 17720 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.364722 16596 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.365240 16743 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:50.366946 16662 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.368105 17713 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.368415 17719 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.369906 17713 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.369166 17650 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.372849 17416 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.376775 17710 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.380298 17420 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.386485 17651 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.388778 16665 raft_consensus.cc:2955] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.396080 16668 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.425625 16492 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.427482 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 446649301a1c45808dccfebfadfed54c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:50.443984 16491 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:17:50.453660 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 43552.9us
I20260709 13:17:50.466676 17420 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.487619 16588 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.495692 16665 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.531917 17741 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.537106 17415 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.625618 16492 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.644501 16592 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.646665 16745 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.667685 16588 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.687220 16494 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.689836 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet 446649301a1c45808dccfebfadfed54c with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:50.696692 17745 tablet_replica.cc:333] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:50.697929 17745 raft_consensus.cc:2243] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:50.698769 17745 raft_consensus.cc:2272] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:50.702966 17745 ts_tablet_manager.cc:1916] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:50.713577 16745 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.720371 17745 ts_tablet_manager.cc:1929] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:50.720839 17745 log.cc:1199] T 446649301a1c45808dccfebfadfed54c P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/446649301a1c45808dccfebfadfed54c
I20260709 13:17:50.725932 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet 446649301a1c45808dccfebfadfed54c (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:50.729341 16668 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.732529 16745 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:50.733322 17415 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.736501 17739 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.739318 17415 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.740588 17727 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.740885 16592 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.744275 17737 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.747377 17420 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.748044 17739 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.750522 17737 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.753070 17420 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.750643 17415 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.755702 16668 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.750555 17727 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.761345 17419 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.759858 17651 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.769411 16494 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.786186 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet f919f8e7484c4f359aedc6ac05b45076 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:50.786665 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f919f8e7484c4f359aedc6ac05b45076 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:50.787246 17745 tablet_replica.cc:333] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:50.786185 17650 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:17:50.794359 16589 raft_consensus.cc:2955] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.796123 16592 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:50.798059 17741 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:50.799526 16494 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.801251 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:50.802593 16596 raft_consensus.cc:2955] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.812724 16667 raft_consensus.cc:2955] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:50.816767 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 79e87765622246cc907a538b35c52c64 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:50.825039 17745 raft_consensus.cc:2243] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:50.825847 17745 raft_consensus.cc:2272] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:50.829545 17745 ts_tablet_manager.cc:1916] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:50.837800 16492 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:50.848088 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet 79e87765622246cc907a538b35c52c64 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:50.853734 17745 ts_tablet_manager.cc:1929] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:50.854303 17745 log.cc:1199] T f919f8e7484c4f359aedc6ac05b45076 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/f919f8e7484c4f359aedc6ac05b45076
I20260709 13:17:50.855790 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet f919f8e7484c4f359aedc6ac05b45076 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:50.857259 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:50.882755 17746 tablet_replica.cc:333] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:50.883982 17747 tablet_replica.cc:333] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:50.884096 17746 raft_consensus.cc:2243] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:50.884821 17746 raft_consensus.cc:2272] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:50.884853 17747 raft_consensus.cc:2243] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:50.886034 17747 raft_consensus.cc:2272] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:50.887861 17746 ts_tablet_manager.cc:1916] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:50.888598 17747 ts_tablet_manager.cc:1916] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:50.900593 17747 ts_tablet_manager.cc:1929] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:50.900946 17747 log.cc:1199] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/2dc3ee5aa5844e5e9c7e3f566e36a561
I20260709 13:17:50.901131 17746 ts_tablet_manager.cc:1929] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:50.901567 17746 log.cc:1199] T 79e87765622246cc907a538b35c52c64 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/79e87765622246cc907a538b35c52c64
I20260709 13:17:50.902490 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 2dc3ee5aa5844e5e9c7e3f566e36a561 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:50.903201 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet 79e87765622246cc907a538b35c52c64 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:51.927276 16596 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:17:51.936160 16561 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d02ec9cc775248dfa8321ae141d0ad37. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.936959 16743 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:51.938210 17715 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:51.938856 16667 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:51.940850 17750 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:51.948805 17715 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:51.950414 16743 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:51.953378 16668 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:51.969756 16745 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:51.972785 16493 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:51.985085 16710 consensus_peers.cc:597] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b20d43e167554c06a6ae722c1a39c5d3. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:51.988876 16668 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:51.989305 16596 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:51.990828 17650 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:51.991547 17651 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:52.001533 17748 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.005925 16668 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.013345 16596 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.023011 16743 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.024623 16492 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.034624 16668 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:52.036913 17650 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:52.038657 16710 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ecb93faa15cf4f7b853b030dfdc6af28. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:52.046306 16596 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:52.047695 17762 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:52.049741 17651 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.051383 16667 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.055529 16596 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.062852 16743 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.069993 16491 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:52.083490 16710 consensus_peers.cc:597] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3386b3f36f454834aaf359817c1adadb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:52.092751 16667 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:52.094188 17748 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:52.095924 16596 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:52.108276 17748 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:52.114377 17762 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.116055 16667 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.119269 16596 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.131049 16493 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:52.447276 17770 ts_tablet_manager.cc:933] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:52.449836 17770 tablet_copy_client.cc:323] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:52.450590 17772 ts_tablet_manager.cc:933] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:52.451639 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet b20d43e167554c06a6ae722c1a39c5d3 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:52.452327 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet b20d43e167554c06a6ae722c1a39c5d3 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-b20d43e167554c06a6ae722c1a39c5d3
I20260709 13:17:52.453224 17772 tablet_copy_client.cc:323] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:52.455260 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet ecb93faa15cf4f7b853b030dfdc6af28 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:52.455750 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet ecb93faa15cf4f7b853b030dfdc6af28 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-ecb93faa15cf4f7b853b030dfdc6af28
I20260709 13:17:52.469503 16754 tablet_copy_source_session.cc:215] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.461073 16755 tablet_copy_source_session.cc:215] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.475062 17772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecb93faa15cf4f7b853b030dfdc6af28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.506309 17770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b20d43e167554c06a6ae722c1a39c5d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.532895 17772 tablet_copy_client.cc:806] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.533597 17772 tablet_copy_client.cc:670] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.540118 17770 tablet_copy_client.cc:806] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.540802 17770 tablet_copy_client.cc:670] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.542086 17772 tablet_copy_client.cc:538] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.545028 17770 tablet_copy_client.cc:538] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.551951 17772 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:52.565212 17770 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:52.566497 17774 ts_tablet_manager.cc:933] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:17:52.575764 17772 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.576447 17774 tablet_copy_client.cc:323] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:17:52.578382 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet d02ec9cc775248dfa8321ae141d0ad37 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:17:52.578787 17772 tablet_bootstrap.cc:492] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:52.578961 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet d02ec9cc775248dfa8321ae141d0ad37 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-d02ec9cc775248dfa8321ae141d0ad37
I20260709 13:17:52.579571 17772 ts_tablet_manager.cc:1403] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.004s
I20260709 13:17:52.582598 17772 raft_consensus.cc:359] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.583554 17772 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:52.584240 17772 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.594696 17772 ts_tablet_manager.cc:1434] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 13:17:52.596840 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-ecb93faa15cf4f7b853b030dfdc6af28 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:52.597419 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-ecb93faa15cf4f7b853b030dfdc6af28 on tablet ecb93faa15cf4f7b853b030dfdc6af28 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:52.599345 16606 tablet_copy_source_session.cc:215] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.606598 17774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d02ec9cc775248dfa8321ae141d0ad37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:52.612877 17772 ts_tablet_manager.cc:933] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:52.630049 17770 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.630565 17772 tablet_copy_client.cc:323] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:52.631323 17770 tablet_bootstrap.cc:492] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:52.632149 17770 ts_tablet_manager.cc:1403] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.014s
I20260709 13:17:52.633090 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 3386b3f36f454834aaf359817c1adadb from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:52.633702 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 3386b3f36f454834aaf359817c1adadb from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-3386b3f36f454834aaf359817c1adadb
I20260709 13:17:52.635517 17770 raft_consensus.cc:359] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.636492 17770 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:52.637230 17770 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.642450 17770 ts_tablet_manager.cc:1434] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:17:52.644766 16754 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-b20d43e167554c06a6ae722c1a39c5d3 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:52.645370 16754 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-b20d43e167554c06a6ae722c1a39c5d3 on tablet b20d43e167554c06a6ae722c1a39c5d3 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:52.652086 17774 tablet_copy_client.cc:806] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.652807 17774 tablet_copy_client.cc:670] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.657491 17774 tablet_copy_client.cc:538] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.660849 16755 tablet_copy_source_session.cc:215] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:52.671250 17774 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:52.674322 17772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3386b3f36f454834aaf359817c1adadb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:17:52.685652 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 97560.9us
I20260709 13:17:52.713788 17772 tablet_copy_client.cc:806] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:52.714486 17772 tablet_copy_client.cc:670] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:52.719760 17774 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.721640 17774 tablet_bootstrap.cc:492] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:52.721890 17772 tablet_copy_client.cc:538] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:52.722640 17774 ts_tablet_manager.cc:1403] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.052s	user 0.031s	sys 0.008s
I20260709 13:17:52.725845 17774 raft_consensus.cc:359] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.726684 17774 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:52.727438 17774 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.732579 17774 ts_tablet_manager.cc:1434] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:17:52.735122 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-d02ec9cc775248dfa8321ae141d0ad37 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:17:52.735193 17772 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:52.735613 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-d02ec9cc775248dfa8321ae141d0ad37 on tablet d02ec9cc775248dfa8321ae141d0ad37 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:52.758267 17772 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:52.759191 17772 tablet_bootstrap.cc:492] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:52.759860 17772 ts_tablet_manager.cc:1403] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.007s
I20260709 13:17:52.762647 17772 raft_consensus.cc:359] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.763468 17772 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:52.764068 17772 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:52.766355 17772 ts_tablet_manager.cc:1434] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:17:52.768292 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-3386b3f36f454834aaf359817c1adadb received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:52.768703 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-3386b3f36f454834aaf359817c1adadb on tablet 3386b3f36f454834aaf359817c1adadb with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:52.982199 17419 raft_consensus.cc:1217] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:52.997996 17419 raft_consensus.cc:1217] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:53.036860 17419 raft_consensus.cc:1217] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:53.159171 17419 raft_consensus.cc:1217] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:53.385011 17762 raft_consensus.cc:1064] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:53.387669 17762 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.395509 17419 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.395751 16667 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.396947 16596 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.397070 17762 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.398463 17651 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.399335 17748 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.420208 17753 raft_consensus.cc:1064] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:53.425088 17753 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.432338 17651 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.439574 16596 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.449651 16668 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.451848 17753 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.458159 17419 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.463209 17419 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.467136 16743 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.474960 17784 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.477921 17784 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.540223 16668 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.550341 16493 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.592905 17791 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.596685 16667 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.609568 16743 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.612411 17651 raft_consensus.cc:1064] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:53.614976 17651 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.635013 16745 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.643685 17419 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:53.645843 17748 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:53.651943 16668 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:53.654326 17796 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:53.658536 16494 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.664220 17762 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.684731 16667 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.690239 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b20d43e167554c06a6ae722c1a39c5d3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:53.678608 17420 raft_consensus.cc:2955] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.726089 17420 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.728845 16492 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:53.739993 17767 raft_consensus.cc:1064] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:53.743070 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet b20d43e167554c06a6ae722c1a39c5d3 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:53.745296 17767 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.773799 17804 tablet_replica.cc:333] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:53.779490 16596 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.787070 17419 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.791215 16668 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.794238 17419 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:53.795207 16667 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.799856 16589 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.785607 17804 raft_consensus.cc:2243] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:53.800112 16596 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.802958 17804 raft_consensus.cc:2272] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:53.803779 17803 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:17:53.800843 17651 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:17:53.801106 17748 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:17:53.800299 17419 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:53.816861 17796 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:53.809574 17762 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:17:53.818225 17802 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.820384 17651 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:53.833950 17804 ts_tablet_manager.cc:1916] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:53.870419 17415 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.876063 17784 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.882795 16743 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:53.881349 17651 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.886238 17416 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.887689 17753 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:53.887135 17796 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.893540 16743 raft_consensus.cc:2955] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.901124 16492 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.904337 17420 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:17:53.920009 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57532.4us
I20260709 13:17:53.926640 16494 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.932920 16596 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.932926 17804 ts_tablet_manager.cc:1929] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:53.937510 17804 log.cc:1199] T b20d43e167554c06a6ae722c1a39c5d3 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/b20d43e167554c06a6ae722c1a39c5d3
I20260709 13:17:53.939538 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet b20d43e167554c06a6ae722c1a39c5d3 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:53.943229 16592 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.967970 16665 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.970363 16668 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.971545 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d02ec9cc775248dfa8321ae141d0ad37 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:53.966717 16494 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.976619 16742 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:53.982419 17415 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:53.984045 17651 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:53.988878 16592 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:53.992229 17796 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.994732 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet d02ec9cc775248dfa8321ae141d0ad37 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:53.995409 17651 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:54.001067 16592 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:53.998044 17420 raft_consensus.cc:2955] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:54.018446 17816 tablet_replica.cc:333] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:54.019496 17816 raft_consensus.cc:2243] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:54.021741 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ecb93faa15cf4f7b853b030dfdc6af28 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:54.027510 16491 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:54.032130 17816 raft_consensus.cc:2272] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:54.031970 16743 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:54.038022 17816 ts_tablet_manager.cc:1916] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:54.038626 16665 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:54.040354 17748 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:17:54.040548 16480 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet ecb93faa15cf4f7b853b030dfdc6af28 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 3 but the committed config has opid_index of 4
I20260709 13:17:54.043892 17420 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:54.046573 17748 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:54.047678 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet ecb93faa15cf4f7b853b030dfdc6af28 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:54.054261 17807 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:54.056654 17416 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:54.058390 17816 ts_tablet_manager.cc:1929] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:54.058789 17816 log.cc:1199] T d02ec9cc775248dfa8321ae141d0ad37 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/d02ec9cc775248dfa8321ae141d0ad37
I20260709 13:17:54.059964 16665 raft_consensus.cc:2955] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:54.064739 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3386b3f36f454834aaf359817c1adadb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:54.070406 17816 tablet_replica.cc:333] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:54.070508 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet d02ec9cc775248dfa8321ae141d0ad37 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:54.076115 17816 raft_consensus.cc:2243] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:54.080685 17816 raft_consensus.cc:2272] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:54.081923 16493 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:54.084935 17816 ts_tablet_manager.cc:1916] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:54.093942 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 3386b3f36f454834aaf359817c1adadb with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:54.094902 17804 tablet_replica.cc:333] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:54.095855 17804 raft_consensus.cc:2243] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:54.096426 17804 raft_consensus.cc:2272] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:54.099372 17804 ts_tablet_manager.cc:1916] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:54.100741 17816 ts_tablet_manager.cc:1929] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:54.101235 17816 log.cc:1199] T ecb93faa15cf4f7b853b030dfdc6af28 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/ecb93faa15cf4f7b853b030dfdc6af28
I20260709 13:17:54.102722 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet ecb93faa15cf4f7b853b030dfdc6af28 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:54.112061 17804 ts_tablet_manager.cc:1929] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:54.112715 17804 log.cc:1199] T 3386b3f36f454834aaf359817c1adadb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/3386b3f36f454834aaf359817c1adadb
I20260709 13:17:54.114238 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 3386b3f36f454834aaf359817c1adadb (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:55.163880 16668 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:55.170789 16592 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:55.171592 16742 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:55.172209 16636 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 42839b9fe9ae47a5a548348ea9b4eee1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.172793 17814 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:55.174189 17740 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:55.180743 17814 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.182428 16592 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.184190 16742 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.209043 16665 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:55.205250 16492 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.218369 16592 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:55.219314 16742 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:55.220602 17818 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:17:55.225257 16636 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb360c38974b4352a4febd1e0aada45d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.226179 17814 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:55.240796 17814 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.243757 16592 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.252812 16742 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.268781 16743 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:55.275976 16667 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:55.277127 16592 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:55.277504 17748 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:55.278352 16710 consensus_peers.cc:597] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f71532fa5e704f9590387cd12f98666b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.279271 17748 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:55.283774 16493 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.301837 17762 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.304459 16595 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.305825 16667 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.321024 16742 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:55.323529 16491 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.328539 16595 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:55.328539 16667 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:55.329591 16710 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: be6c4adb5a2a4a34b6c89e15242b20a1. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:55.330147 17762 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:55.330976 17807 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:55.341416 17807 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.343043 16592 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.359128 16492 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.363837 16667 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:55.618002 17837 ts_tablet_manager.cc:933] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:55.620122 17837 tablet_copy_client.cc:323] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:55.621883 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 42839b9fe9ae47a5a548348ea9b4eee1 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:55.622473 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 42839b9fe9ae47a5a548348ea9b4eee1 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-42839b9fe9ae47a5a548348ea9b4eee1
I20260709 13:17:55.675283 16681 tablet_copy_source_session.cc:215] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:55.680388 17837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42839b9fe9ae47a5a548348ea9b4eee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:55.696487 17837 tablet_copy_client.cc:806] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:55.697329 17837 tablet_copy_client.cc:670] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:55.707041 17837 tablet_copy_client.cc:538] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:55.719738 17837 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:55.770502 17840 ts_tablet_manager.cc:933] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:55.796518 17837 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:55.798092 17837 tablet_bootstrap.cc:492] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:55.799037 17837 ts_tablet_manager.cc:1403] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.080s	user 0.033s	sys 0.019s
I20260709 13:17:55.811473 17841 ts_tablet_manager.cc:933] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:55.812438 17840 tablet_copy_client.cc:323] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:55.858306 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet f71532fa5e704f9590387cd12f98666b from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:55.859233 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet f71532fa5e704f9590387cd12f98666b from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-f71532fa5e704f9590387cd12f98666b
I20260709 13:17:55.860832 17842 ts_tablet_manager.cc:933] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:55.817167 17837 raft_consensus.cc:359] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:55.866379 17837 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:55.877174 17837 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:55.869118 17841 tablet_copy_client.cc:323] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:55.903182 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet bb360c38974b4352a4febd1e0aada45d from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:55.903901 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet bb360c38974b4352a4febd1e0aada45d from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-bb360c38974b4352a4febd1e0aada45d
I20260709 13:17:55.907665 17842 tablet_copy_client.cc:323] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:55.910177 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet be6c4adb5a2a4a34b6c89e15242b20a1 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:55.910766 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet be6c4adb5a2a4a34b6c89e15242b20a1 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-be6c4adb5a2a4a34b6c89e15242b20a1
I20260709 13:17:55.925432 17837 ts_tablet_manager.cc:1434] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.126s	user 0.009s	sys 0.002s
I20260709 13:17:55.931291 16680 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-42839b9fe9ae47a5a548348ea9b4eee1 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:55.931938 16680 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-42839b9fe9ae47a5a548348ea9b4eee1 on tablet 42839b9fe9ae47a5a548348ea9b4eee1 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:55.937669 16755 tablet_copy_source_session.cc:215] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:55.952059 17840 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f71532fa5e704f9590387cd12f98666b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:55.957816 16681 tablet_copy_source_session.cc:215] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:55.979064 17841 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb360c38974b4352a4febd1e0aada45d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:55.975533 16754 tablet_copy_source_session.cc:215] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:55.985064 17842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be6c4adb5a2a4a34b6c89e15242b20a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:55.996397 17841 tablet_copy_client.cc:806] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:55.997133 17841 tablet_copy_client.cc:670] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:56.008013 17841 tablet_copy_client.cc:538] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:56.015640 17842 tablet_copy_client.cc:806] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:56.016335 17842 tablet_copy_client.cc:670] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:56.017151 17841 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:56.020251 17842 tablet_copy_client.cc:538] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:56.021955 17840 tablet_copy_client.cc:806] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:56.022511 17840 tablet_copy_client.cc:670] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:56.028229 17840 tablet_copy_client.cc:538] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:56.039819 17842 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:56.039928 17840 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:56.069358 17841 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:56.070353 17841 tablet_bootstrap.cc:492] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:56.071080 17841 ts_tablet_manager.cc:1403] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.054s	user 0.026s	sys 0.015s
I20260709 13:17:56.076772 17841 raft_consensus.cc:359] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:56.077732 17841 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:56.085098 17841 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:56.101236 17841 ts_tablet_manager.cc:1434] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:17:56.103111 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-bb360c38974b4352a4febd1e0aada45d received from {username='slave'} at 127.0.0.1:38524
I20260709 13:17:56.103529 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-bb360c38974b4352a4febd1e0aada45d on tablet bb360c38974b4352a4febd1e0aada45d with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:56.112491 17842 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:56.113452 17842 tablet_bootstrap.cc:492] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:56.114933 17842 ts_tablet_manager.cc:1403] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.077s	user 0.034s	sys 0.000s
I20260709 13:17:56.117233 17842 raft_consensus.cc:359] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:56.118069 17842 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:56.118849 17842 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:56.124624 17842 ts_tablet_manager.cc:1434] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:17:56.127381 17840 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:56.127432 16754 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-be6c4adb5a2a4a34b6c89e15242b20a1 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:56.128098 16754 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-be6c4adb5a2a4a34b6c89e15242b20a1 on tablet be6c4adb5a2a4a34b6c89e15242b20a1 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:56.128826 17840 tablet_bootstrap.cc:492] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:56.129633 17840 ts_tablet_manager.cc:1403] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.090s	user 0.025s	sys 0.015s
I20260709 13:17:56.132762 17840 raft_consensus.cc:359] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:56.133579 17840 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:56.134100 17840 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:56.136478 17840 ts_tablet_manager.cc:1434] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:56.139600 16754 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-f71532fa5e704f9590387cd12f98666b received from {username='slave'} at 127.0.0.1:41680
I20260709 13:17:56.140022 16754 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-f71532fa5e704f9590387cd12f98666b on tablet f71532fa5e704f9590387cd12f98666b with peer ba2df6a668144022a223e504ae571f21
I20260709 13:17:56.160836 17415 raft_consensus.cc:1217] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:56.203876 17415 raft_consensus.cc:1217] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:56.300632 17416 raft_consensus.cc:1217] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:56.357000 17416 raft_consensus.cc:1217] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:17:56.657174 17845 raft_consensus.cc:1064] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:56.659474 17845 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:56.665858 17416 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.666826 16739 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.667618 16595 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.669106 17818 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:56.670070 17845 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:56.676061 17740 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:56.693476 17818 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.697108 16739 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.698738 16592 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.707106 17416 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.725589 16492 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.727617 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:56.738847 17845 raft_consensus.cc:1064] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:56.771310 17748 raft_consensus.cc:1064] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:56.773684 17748 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:56.754276 17845 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:56.791640 17416 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.797258 17807 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:56.803287 16592 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.805763 16667 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.805559 17807 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:56.808120 17762 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:56.844635 16595 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.845063 17416 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.846894 17818 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:56.847638 16595 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.849298 16742 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:56.850244 17814 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:56.852819 16667 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.853238 16665 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:56.853232 17857 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:17:56.841480 17807 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.861539 17420 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.863508 17814 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.864157 16593 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.873553 17857 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.880802 17818 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.884320 17420 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.893889 17415 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.909024 16593 raft_consensus.cc:2955] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.908351 16493 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:56.945869 16743 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:56.952214 17857 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.959487 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 42839b9fe9ae47a5a548348ea9b4eee1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:56.959728 16491 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.963471 16742 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.976549 16665 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.976722 17416 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:56.978044 16596 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.979236 17835 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:56.980365 17762 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:56.988587 17762 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.989626 16667 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:56.990828 17748 raft_consensus.cc:1064] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:17:56.993690 17748 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.008354 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f71532fa5e704f9590387cd12f98666b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:57.013119 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet 42839b9fe9ae47a5a548348ea9b4eee1 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:57.013543 17416 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.033643 17871 tablet_replica.cc:333] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:57.037835 17871 raft_consensus.cc:2243] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.027164 16491 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.042344 16667 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:57.043251 17871 raft_consensus.cc:2272] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.055328 16493 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.053495 17762 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
W20260709 13:17:57.070302 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 72294.9us
I20260709 13:17:57.084168 17415 raft_consensus.cc:2955] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.043815 16593 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:57.091291 17420 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:17:57.091872 17762 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:57.107121 17871 ts_tablet_manager.cc:1916] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:57.108273 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet f71532fa5e704f9590387cd12f98666b with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:38574
I20260709 13:17:57.108135 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:57.110775 16668 raft_consensus.cc:606] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:17:57.115087 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:57.115689 16667 raft_consensus.cc:606] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:17:57.120740 16668 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.122104 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:57.126811 17762 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:57.140290 16667 raft_consensus.cc:606] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:17:57.152195 17845 raft_consensus.cc:993] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:57.152922 17845 raft_consensus.cc:1081] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:57.154513 17415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:17:57.155297 17415 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.155622 17415 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.160431 17416 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:57.163519 16593 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:57.169729 17415 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.170900 17740 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:57.165436 17762 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.188412 17869 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:57.192356 16596 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.173869 16665 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.202044 17814 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.207612 17857 raft_consensus.cc:993] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:57.208151 17818 raft_consensus.cc:1081] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:57.211213 16493 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.214303 17415 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:57.219926 17416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:17:57.224625 17416 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.203068 16593 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.225106 17416 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.218827 17877 tablet_replica.cc:333] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:17:57.220618 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:57.230345 16667 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.230717 16667 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:57.230468 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bb360c38974b4352a4febd1e0aada45d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:57.231532 16667 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.233705 16667 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.219434 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:57.235145 16592 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.238746 17420 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.232422 17871 ts_tablet_manager.cc:1929] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:57.254343 17871 log.cc:1199] T 42839b9fe9ae47a5a548348ea9b4eee1 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/42839b9fe9ae47a5a548348ea9b4eee1
I20260709 13:17:57.258097 16592 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:57.260421 17877 raft_consensus.cc:2243] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.261355 17877 raft_consensus.cc:2272] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.263098 17416 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.265264 16743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:39096
I20260709 13:17:57.265865 16743 raft_consensus.cc:606] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:57.265743 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet 42839b9fe9ae47a5a548348ea9b4eee1 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:57.267168 16667 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:57.266741 16493 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.280158 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:17:57.281423 17416 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:57.295794 16742 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.298890 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:57.299463 16667 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.299751 16667 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:57.299808 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:57.300366 16665 raft_consensus.cc:606] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:57.300526 16667 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.301821 16667 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.294463 17877 ts_tablet_manager.cc:1916] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:57.291214 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:57.306613 16743 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.307394 16595 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:57.309059 17870 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:57.311546 16667 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:57.281316 17847 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.312461 17416 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:17:57.315248 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:17:57.315459 17416 raft_consensus.cc:2955] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.318347 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:57.318967 16667 raft_consensus.cc:606] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:57.321942 16743 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:57.324023 17870 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:57.333817 17847 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:17:57.334498 17880 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.335072 17880 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:17:57.334856 17847 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.357227 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:17:57.358279 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet bb360c38974b4352a4febd1e0aada45d with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37620
I20260709 13:17:57.358896 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:57.359956 17871 tablet_replica.cc:333] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:17:57.367419 17748 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.369697 16593 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.368352 17814 raft_consensus.cc:993] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:57.373247 17814 raft_consensus.cc:1081] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:57.375396 17420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:17:57.375867 17420 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.376171 17420 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.382211 17420 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.391793 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:17:57.392340 16588 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.392683 17420 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:57.403277 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:57.403865 16667 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.403954 17871 raft_consensus.cc:2243] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.404176 16667 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:57.404951 17871 raft_consensus.cc:2272] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.404898 16667 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.406005 16667 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.407791 16588 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:57.414939 17880 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.413115 17871 ts_tablet_manager.cc:1916] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:57.415797 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:17:57.412016 16493 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.419870 17881 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.420667 17881 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:17:57.419816 17419 raft_consensus.cc:2955] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:17:57.421741 17881 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:57.437264 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet be6c4adb5a2a4a34b6c89e15242b20a1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:17:57.442058 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:57.468357 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet be6c4adb5a2a4a34b6c89e15242b20a1 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:36070
I20260709 13:17:57.496100 16491 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.519353 17877 ts_tablet_manager.cc:1929] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:57.519956 17845 raft_consensus.cc:993] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:57.519969 17877 log.cc:1199] T f71532fa5e704f9590387cd12f98666b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/f71532fa5e704f9590387cd12f98666b
I20260709 13:17:57.523059 17845 raft_consensus.cc:1081] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:57.525322 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:17:57.525875 16593 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.526074 17885 tablet_replica.cc:333] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:17:57.526265 16593 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.527432 17885 raft_consensus.cc:2243] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.528903 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet f71532fa5e704f9590387cd12f98666b (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:57.528116 17885 raft_consensus.cc:2272] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.533258 17885 ts_tablet_manager.cc:1916] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:17:57.535745 16491 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.539335 17818 raft_consensus.cc:993] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:57.542552 17818 raft_consensus.cc:1081] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:57.541730 16593 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.544175 16595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:17:57.544677 16595 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.544994 16595 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.546653 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:57.547241 16667 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.547504 16667 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:57.548300 16667 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.549299 16667 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.550813 17871 ts_tablet_manager.cc:1929] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:57.551349 17871 log.cc:1199] T bb360c38974b4352a4febd1e0aada45d P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/bb360c38974b4352a4febd1e0aada45d
I20260709 13:17:57.551517 16595 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.552876 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet bb360c38974b4352a4febd1e0aada45d (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:57.554201 16595 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:57.554373 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:57.554927 16665 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:57.555212 16665 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:57.555727 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:57.556260 16743 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.562793 17885 ts_tablet_manager.cc:1929] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:17:57.565459 17885 log.cc:1199] T be6c4adb5a2a4a34b6c89e15242b20a1 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/be6c4adb5a2a4a34b6c89e15242b20a1
I20260709 13:17:57.565796 16593 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:57.568058 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:57.555898 16665 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.568615 16745 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.569020 16665 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:57.563305 16491 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:57.562552 16667 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:57.575198 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet be6c4adb5a2a4a34b6c89e15242b20a1 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:17:57.576299 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:57.576324 16745 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:57.578053 17784 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.581103 17784 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:57.581933 17784 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.562311 16743 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:17:57.584345 16665 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:17:57.585717 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:57.586704 17851 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:57.587350 17851 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:57.588387 17851 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.596006 16491 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.610813 16491 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 3, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.807858 16667 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.808879 17886 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.814847 17748 raft_consensus.cc:993] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:57.815397 17748 raft_consensus.cc:1081] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:57.817790 16595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38582
I20260709 13:17:57.818300 16595 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:57.818641 16595 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.823302 16667 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.826740 16593 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.826483 16595 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.829422 17880 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.834219 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:57.834828 16667 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.833541 16595 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:57.830672 17886 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.845813 16667 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:57.845918 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:57.846503 16743 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:57.846827 16743 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:57.847133 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:57.848066 17851 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:57.847673 16743 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.848560 17851 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:57.849272 16743 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:57.849470 17851 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:57.862377 16743 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:57.870223 16494 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:57.872721 16588 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.874106 17886 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.895558 16743 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.897341 17886 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.921051 16668 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:57.922789 17882 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:57.976064 16745 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:57.977196 17851 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:57.992259 16668 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:57.995270 17888 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.067291 16743 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.068544 17784 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.079799 16668 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.081935 17851 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.360070 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:58.383188 16668 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.384479 17905 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.390223 16745 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:58.392184 17905 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.422796 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:58.486290 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:58.486816 16667 raft_consensus.cc:606] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:58.489120 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:58.489562 16667 raft_consensus.cc:606] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:58.491757 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:58.492166 16667 raft_consensus.cc:606] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:58.494165 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:58.494575 16667 raft_consensus.cc:606] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:17:58.496708 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd752e770ac43a799af500f16d446d5"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:17:58.497171 16667 raft_consensus.cc:606] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:58.499622 16743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:39096
I20260709 13:17:58.500080 16743 raft_consensus.cc:606] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:17:58.501185 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:17:58.502575 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:58.556432 17818 raft_consensus.cc:993] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.556960 17818 raft_consensus.cc:1081] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.558889 16595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:17:58.559442 16595 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.559805 16595 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.565475 16595 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.568156 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:58.568786 16667 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.568886 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:58.569118 16667 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:58.569478 16743 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.569891 16667 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.570947 16667 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.568346 16595 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:58.576817 16745 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:58.590237 16743 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.590903 16667 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.592350 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:58.593449 17888 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:58.593907 17888 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:17:58.594388 16710 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc0ed8d0a60640a0a5ce3a772c3eef5c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.594678 16667 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:58.594776 17888 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.595906 17807 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.598088 16588 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:58.601583 17807 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.618088 16668 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.618454 16595 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.620115 16494 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:58.616602 17835 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.635890 16493 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.646848 16668 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:58.658175 17818 raft_consensus.cc:993] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.658825 17845 raft_consensus.cc:1081] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.661357 16595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd752e770ac43a799af500f16d446d5"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:17:58.662026 16595 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.662398 16595 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.665012 16743 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:58.665653 16636 consensus_peers.cc:597] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ada6718f86ea49aa96bb0fdac1acca8e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.665712 16588 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:58.666776 17845 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.667557 17818 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.672981 16595 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.675436 16595 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:58.676721 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:58.677317 16668 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.677599 16668 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:58.678125 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:58.678704 16745 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.678380 16668 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.679719 16668 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.682257 17818 raft_consensus.cc:993] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:58.682767 17818 raft_consensus.cc:1081] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:17:58.685895 17419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:17:58.686455 17419 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.686812 17419 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.689527 16745 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.690789 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:58.690734 17857 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.692086 16668 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.693288 17888 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:58.696439 17888 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:58.697326 17888 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.698477 17419 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:58.700426 17419 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:58.699031 16595 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.701210 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:58.701462 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:58.701864 16668 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.702111 16745 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.702239 16668 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:58.703387 16668 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:58.706676 16743 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.708730 16745 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:58.710021 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:58.711112 17880 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:58.711668 17880 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:17:58.712531 17880 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:17:58.718737 16668 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.718827 16493 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:58.732059 16668 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:17:58.730047 16491 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.740437 16668 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:58.744721 17870 raft_consensus.cc:993] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.745265 17870 raft_consensus.cc:1081] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.747308 16595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38582
I20260709 13:17:58.749326 16595 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.755911 16595 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:58.762727 16595 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.762069 16492 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:17:58.771603 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:58.772179 16667 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:58.775939 16595 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:58.778000 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:58.778506 16743 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:58.778795 16743 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:17:58.779510 16743 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.780622 16743 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:58.787273 16743 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:17:58.785921 16667 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:17:58.789876 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:58.790488 17818 raft_consensus.cc:993] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.790951 17818 raft_consensus.cc:1081] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.790973 17888 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:58.795113 17888 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:58.796203 17818 raft_consensus.cc:993] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.798503 17818 raft_consensus.cc:1081] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:17:58.797286 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:17:58.804768 17888 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.805622 16593 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.806231 16593 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:58.812304 16636 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73089175ee9c465197a1a11e304346be. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:58.813820 16743 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:58.817325 17845 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:58.819728 16592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:17:58.819149 16593 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.820322 16592 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:58.820737 16592 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.823019 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:58.823422 16595 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:58.823980 16668 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.824419 16668 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:58.825394 16668 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.827005 16668 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.827950 17818 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:58.836761 16668 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.836406 17845 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.838331 16593 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:58.839330 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:58.840373 16743 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.840240 16745 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.841632 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:17:58.843978 17784 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:58.844411 17784 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:58.845155 17784 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.847860 16743 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.851955 16592 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.854645 16592 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:58.857975 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:17:58.858734 16667 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:58.859105 16667 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:17:58.866034 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:17:58.866910 16745 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.881287 16745 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.882503 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:17:58.883288 17851 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:58.883855 17851 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:17:58.882763 16595 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.884874 17851 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.903082 16667 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:17:58.906409 16667 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:58.915956 16667 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:17:58.939193 16492 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:58.940583 16494 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 4, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:58.943288 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58183.7us
W20260709 13:17:58.946358 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 71872.6us
I20260709 13:17:58.970590 16667 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:59.012319 16492 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.013480 16593 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:17:59.014020 16492 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.014878 17931 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.015616 16743 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:17:59.017103 16636 consensus_peers.cc:597] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4951b0fadf5b43e49a71f0bb4fb7a85e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:59.021070 17927 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.046125 17919 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:59.049064 16592 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:59.062340 16667 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.064026 17851 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.091020 16492 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 1 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.110786 16742 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.113459 17851 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.113842 16742 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:17:59.200361 16667 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:59.200716 17941 ts_tablet_manager.cc:933] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:17:59.202586 17881 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:59.203948 17941 tablet_copy_client.cc:323] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:17:59.205636 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:17:59.206192 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet fc0ed8d0a60640a0a5ce3a772c3eef5c from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-fc0ed8d0a60640a0a5ce3a772c3eef5c
I20260709 13:17:59.217519 16754 tablet_copy_source_session.cc:215] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.223838 17941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.232599 16667 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.234184 17928 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.239588 16742 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:17:59.244840 17881 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:59.255715 17941 tablet_copy_client.cc:806] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:59.256424 17941 tablet_copy_client.cc:670] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:59.268201 16743 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.270325 17905 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.271591 17941 tablet_copy_client.cc:538] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:17:59.289873 17941 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:17:59.296048 17948 ts_tablet_manager.cc:933] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:59.313864 17948 tablet_copy_client.cc:323] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:59.317117 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet ada6718f86ea49aa96bb0fdac1acca8e from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:59.317720 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet ada6718f86ea49aa96bb0fdac1acca8e from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-ada6718f86ea49aa96bb0fdac1acca8e
I20260709 13:17:59.328337 17950 ts_tablet_manager.cc:933] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:17:59.344440 16681 tablet_copy_source_session.cc:215] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.349582 17948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada6718f86ea49aa96bb0fdac1acca8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:59.356230 17950 tablet_copy_client.cc:323] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:17:59.362629 16743 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.364914 17851 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:59.365209 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 73089175ee9c465197a1a11e304346be from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:17:59.365787 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 73089175ee9c465197a1a11e304346be from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-73089175ee9c465197a1a11e304346be
I20260709 13:17:59.390010 16668 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:59.394449 17928 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:59.400964 17948 tablet_copy_client.cc:806] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:17:59.403090 16743 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:59.413384 16668 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:59.418061 16681 tablet_copy_source_session.cc:215] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:17:59.409354 17941 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:59.421010 17928 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:59.422098 17941 tablet_bootstrap.cc:492] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:17:59.422912 17948 tablet_copy_client.cc:670] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:17:59.432181 17929 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:59.457114 17941 ts_tablet_manager.cc:1403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.167s	user 0.020s	sys 0.011s
I20260709 13:17:59.465771 17941 raft_consensus.cc:359] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:59.466737 17941 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:17:59.467576 17941 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:17:59.470562 17941 ts_tablet_manager.cc:1434] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:17:59.553356 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:00.587754 17952 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.009s	sys 0.000s
W20260709 13:18:00.588344 17952 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.009s	sys 0.000s
W20260709 13:18:00.592829 17378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.004s	sys 0.001s
W20260709 13:18:00.593255 17378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.005s	sys 0.001s
I20260709 13:18:00.594204 17958 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.594376 17952 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.594820 17958 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.594954 17952 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.597097 17952 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.597904 17958 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.612528 17961 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.613215 17961 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.615357 17961 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.615681 17958 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.615839 17962 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.616508 17958 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.616575 17962 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.618932 17958 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.618907 17962 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.622835 17962 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.623457 17962 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:00.624397 16668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.195s	user 0.015s	sys 0.009s
W20260709 13:18:00.624809 16668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.015s	sys 0.009s
I20260709 13:18:00.625377 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 659) took 1198 ms. Trace:
I20260709 13:18:00.625499 17962 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.625694 16668 rpcz_store.cc:276] 0709 13:17:59.426711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.426954 (+   243us) service_pool.cc:224] Handling call
0709 13:17:59.427435 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:17:59.427956 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:17:59.428009 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:17:59.428591 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:18:00.625109 (+1196518us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:00.625136 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:00.625189 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:00.625218 (+    29us) raft_consensus.cc:1685] finished
0709 13:18:00.625224 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:00.625314 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4866,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":1196032,"threads_started":1,"wal-append.queue_time_us":43566,"wal-append.run_cpu_time_us":1918,"wal-append.run_wall_time_us":1003617}
I20260709 13:18:00.652268 17962 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.653148 17962 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.653472 17929 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.654201 17929 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.655372 17962 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.657325 17929 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.659436 17929 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.659950 17929 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:00.661721 17965 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.662122 17929 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.662308 17965 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.662664 17929 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.664134 17966 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.666345 17966 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.666352 17929 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:00.664731 17967 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.669746 17967 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.672232 17967 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.672925 17966 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.673151 17929 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.673832 17962 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.674206 17965 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.674393 17962 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.676218 17970 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.676800 17961 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.676707 17970 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.676806 17962 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.677461 17961 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.678637 17970 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.679301 17960 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.679647 17961 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.679745 17960 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.683295 17960 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.680162 17961 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.683892 17961 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.686043 17961 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.694953 17957 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.695676 17957 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.697858 17957 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.698171 17962 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.698788 17962 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.700968 17962 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.701961 17958 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.702533 17958 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.704584 17958 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.706629 17958 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.707118 17958 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.709285 17958 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.709784 17958 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.710191 17958 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.712937 17958 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:00.718415 17961 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.719776 17961 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.723799 17969 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.726368 17969 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.729172 17969 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.725911 17961 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.729638 17966 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.731097 17961 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.731654 17961 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.734118 17961 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.730355 17966 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.723881 17958 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.736285 17958 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.737072 17966 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.715740 17971 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:00.740017 17971 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.742164 17971 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.745033 17971 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.745712 17971 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:00.748149 17957 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.748705 17957 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.750224 17961 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.750787 17957 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:00.750757 17961 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.751573 17957 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.752606 17965 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.753194 17965 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.754959 17961 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.755348 17965 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.755887 17971 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.757354 17965 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.757987 17966 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.758176 17967 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.758586 17966 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.758737 17967 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.760629 17966 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.760946 17967 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.761091 17966 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.761544 17881 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.761610 17966 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.757943 17965 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.764139 17966 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.765638 17965 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.762210 17881 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.769887 17881 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.779938 17881 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.781088 17881 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.784904 17881 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:00.792667 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 207) took 1374 ms. Trace:
I20260709 13:18:00.793022 16492 rpcz_store.cc:276] 0709 13:17:59.417930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.418068 (+   138us) service_pool.cc:224] Handling call
0709 13:18:00.792582 (+1374514us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1,"spinlock_wait_cycles":24320}
I20260709 13:18:00.795372 17970 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.752125 17957 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.778924 17976 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.795954 17970 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.771407 17971 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.788416 17959 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:00.789312 17929 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.789920 17958 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.790064 17965 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.790025 17980 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.790525 17973 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.797705 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:00.797454 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.819348 16595 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:00.801216 17971 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.774811 17974 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.802917 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.801270 17929 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.802228 17961 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.837523 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.839876 17971 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.812824 17962 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.800940 17969 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.841354 16592 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:00.796373 17976 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.839752 17987 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:00.853915 17984 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.854570 17977 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.812266 17965 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.856325 17988 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.863132 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.831270 17881 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.852133 17958 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.870501 17990 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:00.852556 17971 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:00.891959 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.903558 17989 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.896176 17984 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.904198 17989 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.906071 17984 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:00.848732 17961 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.906595 17984 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.879360 17983 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.911991 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.842471 17974 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.911227 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.892700 17957 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.875290 17967 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.841490 17980 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.872994 17969 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.929755 16592 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:00.896399 17966 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.874451 17962 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.881167 17881 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.902421 17971 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.947021 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.820559 17959 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.951105 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.952344 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.953837 16592 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:00.954406 17989 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.881933 17958 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.848054 17973 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.880990 17987 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.929271 17978 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.963059 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:00.965595 17420 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:00.966889 17994 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.967464 17994 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.969645 17994 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:00.970191 17994 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.970584 17994 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.971652 17996 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.927515 17984 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.929247 17986 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.952307 17966 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:00.965391 17970 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.965718 17989 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.988529 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.950919 17967 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.961737 17973 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.950670 17983 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.989543 17991 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:00.907446 17988 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.958030 17982 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.982092 17957 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.907531 17990 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.997843 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:00.998543 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.895179 17977 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:00.997869 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:00.999495 16592 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:00.991066 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.000227 17972 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.992877 17994 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:00.991215 17976 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:00.994091 17970 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:00.989549 17997 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:00.990796 17989 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.972231 17996 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.999248 17981 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.016798 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.001847 17414 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.019869 17977 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.020152 17958 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.020615 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.017506 17419 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:00.987928 17978 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.021320 16595 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.000334 17957 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:01.012822 17976 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.027967 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.032735 17905 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.007244 17982 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.002564 17991 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.017041 17965 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.047906 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.052486 17415 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.053603 17419 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.012651 17970 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.013936 17997 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:00.995934 17986 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.078816 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.079653 17419 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.080701 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.031087 17981 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.043375 17976 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:01.064363 17962 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:01.061781 17993 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.069479 17969 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.063301 17413 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.131875 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.132711 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.071177 17999 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.134805 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.137085 16662 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.052395 17905 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.142414 17999 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.133055 17993 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.029610 17972 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:01.168095 17998 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.168601 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.175913 16666 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.144099 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.220196 17991 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.222760 17982 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.224347 17978 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.224438 17983 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.168749 17998 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.251915 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.270361 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.230562 17961 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.374334 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.392288 17950 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73089175ee9c465197a1a11e304346be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:01.394718 17991 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.393194 17997 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.389828 17993 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.266718 17985 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.271916 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.355227 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.437225 17991 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.354249 17984 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.271911 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.439210 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.352598 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:01.443429 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.344115 16754 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-fc0ed8d0a60640a0a5ce3a772c3eef5c received from {username='slave'} at 127.0.0.1:41680
I20260709 13:18:01.267043 17979 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.443315 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.440109 18004 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.445837 18001 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:01.464792 16754 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-fc0ed8d0a60640a0a5ce3a772c3eef5c on tablet fc0ed8d0a60640a0a5ce3a772c3eef5c with peer ba2df6a668144022a223e504ae571f21
I20260709 13:18:01.439906 17990 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.446530 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.471752 18001 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:01.446686 17996 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.446987 17986 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.447321 17972 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:01.443041 16593 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.351395 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.439601 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.268193 17981 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.231033 17881 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.267920 17999 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.269953 17967 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.270047 17905 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.447796 17980 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.437201 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.451078 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.505128 18004 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.529408 18004 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.458459 16742 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.433012 17988 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.459283 16591 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.443354 17971 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:01.458302 17998 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.461236 16666 raft_consensus.cc:606] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:01.461818 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.467298 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.470294 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.471123 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.465013 16589 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.507500 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.550696 16662 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.551798 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.502856 16670 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.513168 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.513728 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.270280 17966 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.562769 16669 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.564309 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.566895 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.572729 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.574066 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.574699 17959 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:01.578817 17991 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.542114 17981 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.579594 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.580061 17981 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.580560 16670 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.582264 17981 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.540064 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.471470 17979 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.547892 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.522452 16590 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.558375 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.593883 16742 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.455431 17985 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:01.600584 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.602828 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.604051 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.606027 16667 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:01.614701 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.615386 16671 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.604909 16591 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:01.589208 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.572778 18001 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:01.759873 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 170710us
I20260709 13:18:01.757064 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:01.759187 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.638229 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.765996 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.655391 16742 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:01.775106 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.776163 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.776770 16665 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.749050 18004 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:01.748974 17985 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:01.791668 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:01.674623 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002dea80 after lost signal to thread 15479
I20260709 13:18:01.723753 17991 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.786386 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.748745 17994 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.699582 17981 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.628893 17948 tablet_copy_client.cc:538] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:01.798334 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.748821 17979 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.772271 18012 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.771018 18001 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.800247 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:01.772557 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45441.4us
I20260709 13:18:01.806789 18001 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.807220 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.801671 17980 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.808276 17980 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.808676 17980 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.808915 18001 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.805004 17994 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.810966 17960 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.811372 17960 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.811674 18010 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.811969 17960 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:01.812351 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002df160 after lost signal to thread 16528
I20260709 13:18:01.812273 18010 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:01.813638 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08001e85a0 after lost signal to thread 16620
I20260709 13:18:01.802421 18013 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.815119 18013 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.815559 18013 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.815249 17981 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.777431 17959 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.811307 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.636387 17881 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.772073 17990 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.805212 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.803227 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.811470 17994 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.802238 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.831259 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.833001 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.815138 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.792448 17973 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:01.794157 18004 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.832808 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.820997 18012 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.834847 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.839179 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.844949 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.799281 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.793906 17971 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.803140 17991 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.820700 17959 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.854840 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.855237 17959 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.855772 17974 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.856804 17974 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.857147 17974 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.857550 17974 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.838594 17973 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.823952 17881 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:01.859752 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 60517.8us
I20260709 13:18:01.860738 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.825309 17990 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:01.862524 17991 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:01.814728 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc420 after lost signal to thread 16695
W20260709 13:18:01.864701 17443 debug-util.cc:398] Leaking SignalData structure 0x7b0800240380 after lost signal to thread 16769
W20260709 13:18:01.866216 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002f8580 after lost signal to thread 17446
W20260709 13:18:01.866592 17443 debug-util.cc:398] Leaking SignalData structure 0x7b0800305380 after lost signal to thread 17447
I20260709 13:18:01.814687 18010 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.828955 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.830133 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.867343 16668 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.831094 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:01.869536 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.025s	sys 0.037s
I20260709 13:18:01.869607 16666 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
W20260709 13:18:01.870151 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.025s	sys 0.037s
W20260709 13:18:01.870543 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 920784us
I20260709 13:18:01.873230 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.842119 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.799711 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:01.822713 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.849371 16671 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.848006 16662 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.838650 17905 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.853096 17966 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.853824 17971 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.882200 17966 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.882318 17971 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.883241 17990 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.861213 17881 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.881119 17988 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:01.885594 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62984.8us
I20260709 13:18:01.881577 17981 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.881681 17905 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.884838 17969 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.894975 17905 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.895020 17969 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.895511 17969 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.879707 17996 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.894742 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.882624 17966 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.887295 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.890489 17987 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:01.892349 17957 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.893690 17993 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.886508 17988 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.840634 18004 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.906104 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.906002 17967 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:01.922504 17961 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.925164 17961 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:01.925614 17961 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.925882 18012 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.907141 17973 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.926730 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.909494 17991 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:01.886933 17443 debug-util.cc:398] Leaking SignalData structure 0x7b0800323b00 after lost signal to thread 17941
I20260709 13:18:01.919247 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.907060 18007 ts_tablet_manager.cc:933] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:01.908746 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.945888 16559 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.903542 17996 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.912744 17986 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.955906 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.957181 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.958781 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.947976 17972 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.924551 17993 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.979041 17993 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.973234 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.973392 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:01.973713 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:01.948786 17965 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:01.978943 17972 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:01.931152 17988 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.963766 17996 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.984171 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:02.000440 17409 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:01.992045 16594 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
W20260709 13:18:02.006214 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 198527us
I20260709 13:18:01.992497 17929 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:01.980041 17986 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.027014 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:02.027441 17986 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:01.991180 17965 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:18:02.009089 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 63265us
W20260709 13:18:02.010484 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 117396us
W20260709 13:18:02.011391 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 208296us
I20260709 13:18:02.016669 17960 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:01.959184 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.035923 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:01.991966 17972 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.028878 16559 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.047302 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:01.954694 17991 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:02.049486 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 125914us
I20260709 13:18:02.050024 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.047940 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.019069 17975 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.071549 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.042974 17965 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.077342 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.047259 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:02.076288 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:02.048766 17929 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.083791 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:02.026628 17376 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:02.089143 18019 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:02.049175 18016 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.070690 17967 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.058020 17975 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.107141 17975 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.098094 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.096755 18018 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.110338 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.115365 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.079479 17978 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:02.117057 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 67093.3us
I20260709 13:18:02.090826 17957 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.085099 17929 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.098387 17952 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.134819 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.130021 18018 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.091769 16587 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:02.116295 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.091275 18001 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:02.095882 17994 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:02.100739 17967 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.108457 17881 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:02.126225 18019 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.135311 17952 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.135834 16736 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:02.140833 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.131042 17960 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.096866 18016 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.188822 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.195091 18020 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:02.201709 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 151287us
I20260709 13:18:02.135926 17929 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.128018 17957 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.137593 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:02.131927 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 108189us
I20260709 13:18:02.251511 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.095479 17988 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.198777 17952 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.250465 17957 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.267982 17989 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.271229 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.282372 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.256426 18007 tablet_copy_client.cc:323] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:18:02.283820 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.288249 17976 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.131448 17978 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.148655 17980 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.186678 17967 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.290395 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.139070 18018 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.320152 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.322433 17948 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:18:02.241631 18016 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.330329 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.348582 17950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48618 (request call id 249) took 3041 ms. Trace:
I20260709 13:18:02.350004 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.335016 17973 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:02.253818 18020 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.148483 18001 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.355767 17420 raft_consensus.cc:1217] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:18:02.366818 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:02.406739 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:02.365993 17978 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:02.408460 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:02.388039 17991 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:02.394161 16636 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.503s (SENT)
I20260709 13:18:02.399592 16668 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.364645 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.386996 17905 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.197449 17994 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:02.348624 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:02.407753 17976 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:02.397365 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64079.4us
I20260709 13:18:02.404505 18020 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.391533 17965 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.423053 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.409030 17970 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.412955 17881 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:02.420884 17959 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.015s
I20260709 13:18:02.404815 17973 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.438738 18024 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:02.429399 18013 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.613s	user 0.019s	sys 0.002s
I20260709 13:18:02.449012 18014 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.449692 18014 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.450191 18014 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.453044 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.444347 18019 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:02.443113 17969 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.547s	user 0.025s	sys 0.011s
I20260709 13:18:02.419796 17996 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:02.470569 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 407644us
W20260709 13:18:02.475250 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81118.9us
I20260709 13:18:02.502292 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:02.439534 18024 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.517351 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:02.518894 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 400308us
I20260709 13:18:02.521297 16634 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.525022 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.437819 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:02.522938 17993 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.531s	user 0.014s	sys 0.009s
I20260709 13:18:02.528227 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.395743 17950 rpcz_store.cc:276] 0709 13:17:59.307568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:59.307724 (+   156us) service_pool.cc:224] Handling call
0709 13:17:59.329000 (+ 21276us) ts_tablet_manager.cc:934] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
0709 13:18:01.268528 (+1939528us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:01.575537 (+307009us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:01.575695 (+   158us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:01.841631 (+265936us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:01.841939 (+   308us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:02.348392 (+506453us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:02.348482 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":297,"tablet-copy.queue_time_us":19923,"thread_start_us":65457,"threads_started":2}
I20260709 13:18:02.502198 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 264) took 1086 ms. Trace:
I20260709 13:18:02.456108 18013 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.440572 17959 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.539249 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:02.521745 17971 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.009s
I20260709 13:18:02.520298 17999 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:02.522193 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.503270 17969 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.560180 18001 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:02.539533 17987 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.564159 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.543728 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.567065 18032 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.549265 17950 tablet_copy_client.cc:806] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:02.545672 17988 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:02.571309 17415 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.579618 16662 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.591475 17950 tablet_copy_client.cc:670] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:02.563827 16669 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:02.545013 17993 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.595261 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.597499 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.562486 17905 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.601213 16634 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:02.564495 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 136866us
I20260709 13:18:02.589061 16670 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.568502 17987 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.611285 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.611446 18022 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.613366 16671 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.616618 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.625157 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:02.557996 17971 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.599545 18015 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.160s
I20260709 13:18:02.628501 17988 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.591786 18032 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.544391 17411 rpcz_store.cc:276] 0709 13:18:01.374453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.402488 (+ 28035us) service_pool.cc:224] Handling call
0709 13:18:02.328123 (+925635us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfe9c 00007f327cabfa5b 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:02.461068 (+132945us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35408640}
I20260709 13:18:02.614629 17987 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.570955 17984 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.724478 16667 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:02.610531 17974 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.752s	user 0.022s	sys 0.000s
I20260709 13:18:02.719825 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.721590 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.628531 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.648900 17965 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.647739 18022 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.732189 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.749769 17974 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.692824 17988 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.696638 18002 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.758674 18029 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:02.773600 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.777295 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.774196 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.782287 18002 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.774369 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.782748 18002 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.782974 16745 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:02.775241 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.775310 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.700508 18032 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.768415 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.778088 16743 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:02.702767 17973 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.789814 17974 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.793540 16738 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:02.758816 18026 raft_consensus.cc:993] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:18:02.767844 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:02.765977 17972 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.714s	user 0.006s	sys 0.006s
W20260709 13:18:02.734519 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 104813us
W20260709 13:18:02.765929 17975 consensus_meta.cc:287] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.658s	user 0.018s	sys 0.014s
W20260709 13:18:02.757402 17966 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.833s	user 0.018s	sys 0.004s
I20260709 13:18:02.794065 18024 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.761242 17994 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:02.825476 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.817935 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.783591 17988 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.803689 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.819329 17409 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:02.780298 18022 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.736052 18001 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 13:18:02.819496 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.833539 17975 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.836436 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.840703 18027 raft_consensus.cc:1081] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
W20260709 13:18:02.853081 17986 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.003s
I20260709 13:18:02.761103 18018 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.847796 17969 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:02.831079 17978 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:02.929893 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.934151 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.935004 16740 raft_consensus.cc:2410] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:02.935441 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.936578 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.937701 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.939020 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.899427 18008 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:02.942056 16662 raft_consensus.cc:2410] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:02.724213 17984 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:02.801355 18029 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:02.843443 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.832156 17959 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:02.758924 18025 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:02.894942 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:02.887213 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 369899us
I20260709 13:18:02.958405 16667 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:02.959429 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:02.960098 16667 raft_consensus.cc:606] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:02.835228 18013 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:02.842309 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:02.830657 18014 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.828791 18016 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:02.923447 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.029s	sys 0.064s
W20260709 13:18:02.967707 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.029s	sys 0.064s
W20260709 13:18:02.968122 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 558852us
I20260709 13:18:02.781921 17957 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:02.832376 17972 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.839258 17979 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
W20260709 13:18:02.939934 18020 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.004s
I20260709 13:18:02.944828 16665 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.926589 17986 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:02.837544 17966 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:02.943315 17967 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.550s	user 0.003s	sys 0.015s
I20260709 13:18:02.943243 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.970851 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:02.971333 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:02.947077 17984 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.007153 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:02.947199 18029 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:02.963295 17993 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.013522 17416 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:02.957540 18008 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.006443 16744 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:03.032879 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.046828 17419 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.049021 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.032197 16738 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:03.011332 18020 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.034471 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.051898 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.053256 17972 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.092880 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.103065 16592 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.103325 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.104089 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.105177 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.106134 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.092880 16596 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.107668 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 58) took 1541 ms. Trace:
I20260709 13:18:03.095376 16593 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.104105 16740 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:03.101657 17987 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:03.108985 17952 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.002s
I20260709 13:18:03.109362 17952 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.109731 16666 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.110308 17420 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.110442 17987 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:03.111160 17961 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.185s	user 0.009s	sys 0.016s
I20260709 13:18:03.099699 16743 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
W20260709 13:18:03.104259 18037 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:18:03.107717 16671 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.096289 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.095175 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.115190 16588 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.043882 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:03.114506 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65556.7us
I20260709 13:18:03.033046 18013 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.043601 17414 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.111546 17961 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.115792 16741 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:03.119799 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.121172 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.054170 17971 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.034884 17957 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.113724 16590 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.136639 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:03.032510 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 190275us
I20260709 13:18:03.036026 17980 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.103092 17974 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.099255 18033 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:03.164860 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45142.5us
W20260709 13:18:03.167326 16560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.067s	sys 0.121s
I20260709 13:18:03.048213 17967 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.171651 16588 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:03.173879 16667 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.120694 18013 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.005873 18025 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.161991 16587 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:03.108187 16669 rpcz_store.cc:276] 0709 13:18:01.566461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.595556 (+ 29095us) service_pool.cc:224] Handling call
0709 13:18:03.107424 (+1511868us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b540033b154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:03.107578 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485656320}
I20260709 13:18:03.213112 17975 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.202430 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.099643 18002 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.168058 18018 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.166530 17881 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:03.239094 16560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.069s	sys 0.121s
I20260709 13:18:03.242039 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.169104 18032 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:03.178270 18033 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:03.233059 16589 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.256255 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.265027 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.265698 16596 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.259008 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:03.261439 16743 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:03.246618 18013 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.275765 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.267529 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 696) took 1192 ms. Trace:
I20260709 13:18:03.268275 17881 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.264026 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1480) took 1064 ms. Trace:
I20260709 13:18:03.265201 18033 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.272045 18038 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:03.272671 17988 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.273617 17972 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:03.274088 16590 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.279718 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.280278 16596 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.290829 16743 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:03.291338 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.281237 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.274521 17975 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:03.283052 16669 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.283210 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.294534 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.283488 18038 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.300096 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.301128 16593 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.301829 16668 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.303185 18020 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.303282 18038 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.304612 17967 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.282436 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.310024 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.300711 16592 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.306034 16667 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.283068 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.280815 16670 rpcz_store.cc:276] 0709 13:18:02.200007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.274306 (+ 74299us) service_pool.cc:224] Handling call
0709 13:18:02.274779 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:02.614014 (+339235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.249572 (+635558us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54001f7054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:03.262014 (+ 12442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676554112}
I20260709 13:18:03.315055 18018 raft_consensus.cc:993] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:18:03.315656 18018 raft_consensus.cc:1081] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:18:03.316782 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.319064 17957 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.319250 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.319523 17957 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.319939 17957 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.307451 17996 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.307695 18025 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.300396 17975 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.281157 16664 rpcz_store.cc:276] 0709 13:18:02.053185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.057697 (+  4512us) service_pool.cc:224] Handling call
0709 13:18:02.127506 (+ 69809us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:02.505010 (+377504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:02.515511 (+ 10501us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:02.571413 (+ 55902us) log.cc:844] Serialized 26 byte log entry
0709 13:18:02.906806 (+335393us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:02.970753 (+ 63947us) log.cc:844] Serialized 14 byte log entry
0709 13:18:03.005810 (+ 35057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.076023 (+ 70213us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:03.177184 (+101161us) raft_consensus.cc:1685] finished
0709 13:18:03.178141 (+   957us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:03.242773 (+ 64632us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2934,"prepare.queue_time_us":42796,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":175652,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":5917,"thread_start_us":613195,"threads_started":3,"wal-append.queue_time_us":371946}
I20260709 13:18:03.330303 17984 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.330018 18033 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.294098 17992 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.330731 17975 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.335547 17992 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.303282 18002 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.285317 18008 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.342908 16664 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.310232 18029 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.449946 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.450641 18029 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.451254 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.324921 18032 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.295516 18014 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.326685 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.455996 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.457454 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.315204 17961 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.334079 16595 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.461992 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.308151 17978 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.334709 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.463131 17978 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.337230 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.464061 16667 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:03.464331 17978 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.465706 17978 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.466200 16667 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:03.338584 17929 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.023s	user 0.011s	sys 0.000s
I20260709 13:18:03.467223 17929 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:03.469733 16588 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.471083 18014 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:03.472534 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 137898us
I20260709 13:18:03.473294 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.474399 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.475193 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.476745 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.477252 16595 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.477494 17420 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:03.478075 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.340881 16669 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.478824 16595 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.458757 16663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:03.479663 16663 raft_consensus.cc:606] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.479674 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.344561 17984 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.280805 17881 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:03.480975 17881 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.481027 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.350008 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.351254 16741 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:03.342751 17957 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.352358 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.294926 17972 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:03.354185 18033 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.496541 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.497560 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.497884 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:03.498281 16663 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:03.355399 18016 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.355314 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.499527 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.357049 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.500226 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:03.501016 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 144120us
I20260709 13:18:03.501554 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.502138 17957 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.502319 16740 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:03.502990 18008 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:03.503598 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.338308 17992 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.504366 18019 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.504719 17992 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.504835 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.363837 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1478) took 1224 ms. Trace:
W20260709 13:18:03.363837 18022 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.520s	user 0.004s	sys 0.004s
I20260709 13:18:03.367975 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.368459 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.512574 17979 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:03.513204 17972 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.513427 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.514308 17972 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.368638 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.370230 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.500337 16669 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.371228 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.371701 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.372351 17410 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.519740 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.520470 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 318) took 2132 ms. Trace:
I20260709 13:18:03.372529 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:18:03.520897 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.521507 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 4951b0fadf5b43e49a71f0bb4fb7a85e from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-4951b0fadf5b43e49a71f0bb4fb7a85e
I20260709 13:18:03.521845 16742 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:03.522795 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.523852 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.524788 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.525875 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.372905 18013 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.373431 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.374686 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.527819 17973 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.528244 17973 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:03.528415 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 153835us
I20260709 13:18:03.529048 17973 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.529906 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.379633 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.527844 16671 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.380617 17419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 316) took 2028 ms. Trace:
I20260709 13:18:03.531076 18032 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:03.531005 17419 rpcz_store.cc:276] 0709 13:18:01.351931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.352071 (+   140us) service_pool.cc:224] Handling call
0709 13:18:01.903510 (+551439us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:02.340064 (+436554us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfe9c 00007f327cabfa5b 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:02.340334 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.092551 (+752217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.380373 (+287822us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b5400573d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:03.380525 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681588736}
I20260709 13:18:03.531838 18028 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.387101 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.532902 17411 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.533294 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.533963 18013 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.389719 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.390394 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.392870 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.393453 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.534849 16592 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.535629 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.536227 16592 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.395668 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:03.356455 17994 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.539918 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.540123 17994 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.541270 16590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:18:03.540936 17994 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.541847 16590 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:03.541980 17994 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.542296 16590 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:03.397517 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.400457 17412 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.404620 18001 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.407181 16591 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.409348 18025 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:03.409742 18038 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.411703 17967 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:03.552191 17967 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.552515 17967 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:03.418778 17966 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.478188 18014 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.338836 17988 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.478613 18029 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.526414 18022 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.475487 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.482654 17420 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.484850 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.559119 16662 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.559352 16667 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.294497 17986 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:03.493649 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 121968us
I20260709 13:18:03.494746 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.491031 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:03.561268 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70309.9us
I20260709 13:18:03.558511 18025 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.500404 17929 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.358143 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.357205 17952 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.502612 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:03.507032 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.505503 18008 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.581521 18014 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:03.581923 18010 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:03.582067 18014 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:03.582384 18010 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.582548 18014 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.508531 18019 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.509547 18015 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.505646 16665 rpcz_store.cc:276] 0709 13:18:02.139765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.313292 (+173527us) service_pool.cc:224] Handling call
0709 13:18:02.323052 (+  9760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.012337 (+689285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.363618 (+351281us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54003df9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:03.363775 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427602816}
I20260709 13:18:03.512465 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:03.369767 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 265) took 1326 ms. Trace:
I20260709 13:18:03.514588 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.519112 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.516062 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.549153 18001 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.374940 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.534083 16594 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.391366 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:03.452912 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.544752 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.545185 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.609721 16743 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.610294 18046 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.610450 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.512609 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.550707 16666 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.612927 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.613636 16671 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.551246 16587 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.619542 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.620620 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.621308 16587 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:03.622411 16594 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:18:03.622956 16594 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:03.623347 16594 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.627846 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.629107 16745 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:03.629228 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.627177 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.630682 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.631497 16681 tablet_copy_source_session.cc:215] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:18:03.631700 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.632337 16736 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:03.633128 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.555086 16591 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.644484 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:03.566359 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.645972 16667 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.645982 17929 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:03.647094 17929 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:03.647509 17929 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:03.648005 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.520872 17410 rpcz_store.cc:276] 0709 13:18:01.387755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:01.408872 (+ 21117us) service_pool.cc:224] Handling call
0709 13:18:02.275192 (+866320us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfe9c 00007f327cabfa5b 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:03.372128 (+1096936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.519615 (+147487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.520245 (+   630us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54005d25d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:03.520388 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541068928}
I20260709 13:18:03.569531 17420 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.558070 17881 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.573267 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:03.652330 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 79130.6us
I20260709 13:18:03.653213 18001 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.573467 17950 tablet_copy_client.cc:538] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:03.654263 18001 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.655438 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.655835 17416 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.656172 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.660571 17420 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.661623 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.663206 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.664307 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.659490 17952 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.662356 18026 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:03.667255 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.668076 17969 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:03.668180 18029 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.668526 17969 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.668977 17969 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:03.669966 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.671027 17996 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:03.671257 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.671482 17996 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:03.671897 17996 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:03.672412 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.673461 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.674455 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.674700 18014 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.675761 17414 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.678515 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.610743 18046 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.679579 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.679702 18046 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.680389 17978 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:03.681156 18046 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.575598 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.547976 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.580741 18022 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:03.581437 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 78913.5us
I20260709 13:18:03.685667 16590 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.507762 16740 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:03.582828 18010 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:03.558769 17979 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.585662 17948 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:03.586647 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:03.581595 17415 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:03.588716 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.591213 17967 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.594669 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1489) took 1175 ms. Trace:
I20260709 13:18:03.530001 17973 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.604857 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.605690 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.604873 16668 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.606756 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.701786 16739 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:39096
I20260709 13:18:03.702301 16739 raft_consensus.cc:606] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:03.616261 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:03.616997 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:03.633368 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.700301 16662 rpcz_store.cc:276] 0709 13:18:02.419479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:03.033079 (+613600us) service_pool.cc:224] Handling call
0709 13:18:03.033520 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:03.559548 (+526028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:03.594359 (+ 34811us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b5400145854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:03.594594 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208328576}
I20260709 13:18:03.640483 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.599865 17409 rpcz_store.cc:276] 0709 13:18:02.043804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:02.057894 (+ 14090us) service_pool.cc:224] Handling call
0709 13:18:03.369560 (+1311666us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b54005d25d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:03.369710 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502599936}
I20260709 13:18:03.627179 16664 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.581868 18008 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.644063 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:03.551402 18038 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.645345 18025 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.574180 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.683571 17881 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.683943 18032 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:03.682432 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.688251 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.688078 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.692436 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.693595 17948 tablet_bootstrap.cc:492] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:18:03.693764 16594 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.705386 16745 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:03.735697 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47921.4us
I20260709 13:18:03.735971 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.736775 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.731529 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.738238 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.740294 16596 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:03.722581 16743 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:03.727744 17969 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.748231 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.720762 18038 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:03.750463 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.752480 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.754050 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.755983 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:03.757555 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.706399 18019 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:03.760102 16592 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:03.760712 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.728721 18032 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.766327 18015 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.763092 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.746788 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.762974 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.759444 17998 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:18:03.775795 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.717599 18045 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:18:03.743567 18022 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.759483 16738 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.757458 18026 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:03.765250 17948 ts_tablet_manager.cc:1403] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 1.443s	user 0.025s	sys 0.013s
I20260709 13:18:03.767527 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.764498 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.780230 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.797648 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.744875 17994 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:18:03.795917 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.767385 18032 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:03.766945 18020 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:03.808095 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.776479 18015 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.833577 16745 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
W20260709 13:18:03.830097 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51592.7us
I20260709 13:18:03.776306 18028 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:03.836670 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56759.2us
I20260709 13:18:03.837939 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.838575 18022 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:03.838734 17961 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:03.839116 16742 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.838073 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.808115 18026 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.843704 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:03.819262 16590 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.846490 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.830900 18020 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:03.831051 17881 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.766516 18038 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.761560 18008 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.859961 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.830578 17992 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
W20260709 13:18:03.865975 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 146942us
I20260709 13:18:03.849359 18026 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:03.868126 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:03.836243 18049 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:18:03.867057 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.879808 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.880287 18008 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.849002 17959 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.883455 17984 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:03.886438 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.886546 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.889878 18026 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.897049 17992 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:03.907509 18038 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.862033 18020 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:03.908416 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.909235 18007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.935408 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.858470 18028 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:03.930841 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 95766.2us
W20260709 13:18:03.931535 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 89686us
I20260709 13:18:03.845376 17929 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:03.932181 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45697.7us
I20260709 13:18:03.942030 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.943612 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.943697 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.928228 17994 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.872715 18015 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.938457 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.929175 18046 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:03.938478 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.938035 16742 raft_consensus.cc:2376] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2. Re-sending same reply.
I20260709 13:18:03.942528 16738 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:03.934536 18054 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.945338 18013 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.958757 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.959523 18026 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.960333 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.961122 16664 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:03.947463 17959 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.963227 17929 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.951385 17986 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.963590 18049 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:03.964308 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.964722 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.965616 18049 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.965785 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.966028 18049 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:03.967262 18015 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.966997 18046 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.967831 18014 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.969189 18046 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:03.972097 17998 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.972681 17998 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:03.972218 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.973480 17998 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:03.971998 16593 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:03.934947 17881 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:03.952016 17969 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:03.931322 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 73934.3us
I20260709 13:18:03.976423 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.001024 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.977417 16742 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:03.979918 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:04.002264 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.002177 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.994947 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.994947 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.995319 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:03.944758 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.950273 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:03.975301 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:03.991392 17969 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:18:04.030151 17409 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:04.030704 17969 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.031062 17969 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:04.031807 17969 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.032714 17969 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.978415 18032 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:03.987962 18048 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:03.950961 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.034799 18048 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:04.035384 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84503.3us
I20260709 13:18:04.027441 16662 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.036177 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.035698 18048 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.036594 18035 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.036929 18038 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.037292 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.035676 18010 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:04.037938 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 93260.9us
I20260709 13:18:04.038259 16739 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:03.931671 16594 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:03.954713 17413 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:18:03.996577 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.956172 17412 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.039328 17961 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:03.955917 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.026958 17978 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.040344 17978 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:04.040413 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 84568.8us
I20260709 13:18:04.040848 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.040719 17996 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.041286 17978 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.041826 16665 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.042390 17978 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:03.946918 17967 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.043210 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.043512 18026 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.043891 18035 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:04.044184 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.044354 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.044536 18035 raft_consensus.cc:2764] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:04.045141 18035 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.045547 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.045981 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.046994 17996 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.047083 18046 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:04.047552 18046 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:04.047917 18046 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.048092 18032 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:04.049237 18038 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.051029 17992 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:03.894409 17948 raft_consensus.cc:359] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:03.992061 18044 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.054924 18044 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.055023 16662 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:03.986903 17881 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:04.055727 18044 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.056329 17881 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.058410 16737 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:04.059442 17948 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
W20260709 13:18:04.059692 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:03.990440 17950 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:18:04.006901 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.006943 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:03.981222 17415 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.064182 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:03.949030 17994 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.052186 17992 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:18:04.056780 18044 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.059352 18054 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:03.986814 18045 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.063637 16736 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:04.066493 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:04.068141 18019 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.039160 18010 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.046551 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.069518 16745 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.070300 18047 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.070465 16662 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.052186 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:03.988610 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.003109 17988 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:03.974843 17998 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.078444 16671 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:04.078627 18045 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.080489 18057 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.081090 17994 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.060354 18015 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:18:04.082490 18008 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:04.082782 18053 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:18:04.060230 17948 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:04.083312 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.083877 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:04.072120 17376 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.084946 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:03.975477 18049 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.086189 16663 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.168385 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.169497 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.071455 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:04.091874 17415 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.089558 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.171360 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81868.6us
I20260709 13:18:04.172034 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.173031 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.114982 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.173777 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.141156 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 89053.3us
I20260709 13:18:04.084353 17992 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.136039 18057 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.136096 17988 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:04.152143 17948 ts_tablet_manager.cc:1434] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.333s	user 0.008s	sys 0.001s
I20260709 13:18:04.153680 18046 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.146638 17980 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.156213 17984 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:04.091167 18048 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.089988 17973 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.138899 18038 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:04.202062 18038 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.087913 18019 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.183385 17992 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:04.178126 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:04.183643 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.089809 18047 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:04.203073 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 131042us
I20260709 13:18:04.178427 18020 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:04.199936 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.211444 18057 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.212237 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:04.212728 16667 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:04.213039 18058 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.212728 18057 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.174980 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.201402 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.176846 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.187541 18054 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.186525 17994 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.220767 17992 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.222002 17992 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.186035 17929 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.194557 18029 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.223521 18029 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:04.224413 18029 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.202168 18059 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.183043 17986 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.227496 18019 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:04.228165 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52640.5us
I20260709 13:18:04.228586 18045 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.229008 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.229746 18045 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.230479 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.220127 18035 raft_consensus.cc:2764] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:04.232957 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.234066 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.230121 18046 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.235551 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.236642 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.238008 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:04.239168 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.239713 17979 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.213500 18058 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.237944 18008 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.246047 17979 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:04.246457 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.238673 17881 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.245814 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.249079 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74192.2us
I20260709 13:18:04.235039 18001 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:04.234776 18059 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.246914 17986 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:18:04.255487 17966 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.252604 16742 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:04.256479 18061 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.256919 18061 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.257270 18061 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.255553 17986 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.238835 17972 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.254434 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.244451 18054 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.259656 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.215090 18038 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.265482 18038 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:04.267455 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.249279 18008 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.213687 17988 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.239785 18015 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:18:04.253401 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.263108 18055 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.264717 18020 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:04.249912 18058 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.279345 17988 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:04.268359 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:04.253099 17979 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:04.271281 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.272753 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.274544 17969 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.271806 18008 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.282513 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:04.276981 18010 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.281492 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.260424 17881 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:04.302033 17881 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.259963 17986 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.304957 17986 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.306231 17986 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.282778 18035 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:18:04.310830 18049 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.284971 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.272295 17972 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.285281 17979 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.292927 18008 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.295187 17969 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:04.273329 18001 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:04.325860 18001 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.296787 17992 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.326256 17966 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.326465 17992 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.326740 17966 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:04.326973 17992 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.327227 17966 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.299015 18019 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.312515 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.332738 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.333096 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.324684 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.326313 18001 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.340765 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.342047 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.300385 18010 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.311959 18020 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.345659 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.344455 18023 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:04.347249 18023 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:04.331259 18045 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.341229 18055 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.258121 18047 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.348690 18023 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.349134 17996 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.347720 17972 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:18:04.349931 18008 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.347833 18032 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.343652 17980 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.351897 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.364316 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.366210 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.345014 16664 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.348325 16737 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:04.375363 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.368675 17972 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.367776 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.371042 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.341346 18054 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:18:04.351022 17973 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.380191 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.360759 18053 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.367722 18026 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.377224 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.380215 16742 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.381824 18008 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.384441 17881 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:04.372103 18045 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.386965 18064 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.387396 18064 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.386281 18055 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.390836 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.393698 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.393726 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.394465 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.380299 17996 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.381018 17972 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.411166 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:04.411945 17415 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:04.396050 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.392771 18025 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:04.419003 18025 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.402486 18032 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.403430 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.406505 17967 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:04.487015 17967 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.419835 18025 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.487871 17967 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.488231 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:04.488803 18025 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:04.488945 17967 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:04.409427 17929 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:04.387773 18064 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.412889 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.413416 17376 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:04.385936 18010 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.417424 17972 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.418246 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:04.393177 18054 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.418551 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:04.424959 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.512040 18001 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:04.455659 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.499063 17994 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:18:04.506153 17952 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.514184 18001 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.514709 17952 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:04.516388 18063 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.516958 18063 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:04.463837 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 67850.2us
I20260709 13:18:04.468060 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.468752 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.522183 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.522722 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.523313 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:04.502010 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88677.7us
I20260709 13:18:04.517964 18029 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:04.524561 18029 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:04.525460 18029 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.523340 16744 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.515653 17952 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.526641 18029 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.471622 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:04.472384 18065 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:18:04.472290 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.472730 17996 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:04.473240 17959 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.474761 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.549508 17996 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.551038 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.553638 16737 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:04.476332 16669 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:04.478114 17966 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:04.480170 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.412413 18053 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:04.564085 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.567142 17966 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.540693 17979 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.568344 16739 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:04.568794 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.569561 17979 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.569702 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.570832 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.571844 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.572765 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.555460 17959 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.569906 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.592572 16737 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.517831 18063 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.396278 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.385385 18020 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.486310 16736 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:04.484673 18061 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.605014 18010 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.606211 18061 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.417559 17992 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.607214 16739 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.608209 17992 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.493691 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.399461 18045 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:18:04.612136 16745 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.490798 17929 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.491916 18007 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48618 (request call id 255) took 3509 ms. Trace:
I20260709 13:18:04.511190 17972 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.614009 18047 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.470728 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.615532 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 144882us
I20260709 13:18:04.536103 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.535650 16741 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.391132 17969 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.542563 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.634987 16744 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:18:04.600039 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 200620us
I20260709 13:18:04.612738 16665 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:04.619274 17419 raft_consensus.cc:1217] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:18:04.623709 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.630514 17881 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.637833 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.639519 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.633517 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:04.590714 16740 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:04.651137 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:04.685786 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 134815us
I20260709 13:18:04.691025 17984 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:04.691720 17984 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:04.692653 17984 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.613564 18007 rpcz_store.cc:276] 0709 13:18:00.982351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:00.982737 (+   386us) service_pool.cc:224] Handling call
0709 13:18:01.906684 (+923947us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cac0785 00007f3284bf5298 00007f3284bf6a77 00007f3284c28f4e 00007f3284c0dfdd 00007f3284c0df89 00007f3284c0df19 00007f3284c0dee1 00007f3284c0d10d 000000000038f354
0709 13:18:01.959693 (+ 53009us) ts_tablet_manager.cc:934] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
0709 13:18:03.796120 (+1836427us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:18:04.156558 (+360438us) tablet_metadata.cc:665] Metadata flushed
0709 13:18:04.156679 (+   121us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:18:04.361434 (+204755us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:18:04.380943 (+ 19509us) tablet_replica.cc:206] Creating consensus instance
0709 13:18:04.411044 (+ 30101us) ts_tablet_manager.cc:989] Replica registered.
0709 13:18:04.491077 (+ 80033us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":111596,"spinlock_wait_cycles":55425920,"tablet-copy.queue_time_us":398369,"thread_start_us":395645,"threads_started":2}
I20260709 13:18:04.694422 18007 tablet_copy_client.cc:806] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:18:04.695055 18007 tablet_copy_client.cc:670] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:18:04.670323 17986 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:04.652055 17992 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.695787 17992 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.695684 17986 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.696264 17992 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.697962 17986 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.626049 17979 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:04.704758 16743 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.707245 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.707897 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-ada6718f86ea49aa96bb0fdac1acca8e received from {username='slave'} at 127.0.0.1:38524
I20260709 13:18:04.708389 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-ada6718f86ea49aa96bb0fdac1acca8e on tablet ada6718f86ea49aa96bb0fdac1acca8e with peer ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.646384 18045 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.719590 17929 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.721712 17929 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.722071 16740 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.722617 16492 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.724534 16492 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.650550 17966 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:04.727582 17966 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.730031 18064 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:04.736111 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.726218 16492 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.737780 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:04.745775 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64042.7us
I20260709 13:18:04.729645 17966 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:04.749657 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.753216 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.751794 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 133207us
I20260709 13:18:04.758728 16742 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:04.751449 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:04.759999 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.755755 16494 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 2 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.745630 18055 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:04.786702 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:04.771283 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.790343 16741 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.791049 17419 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:18:04.775285 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.747514 18010 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.745242 18063 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.795148 18032 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.750900 18045 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.798053 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.799093 16745 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:04.803242 16589 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.804041 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:04.808992 16593 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:04.805536 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.760528 17996 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:04.759255 18020 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.815878 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.793352 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.819272 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.790838 18061 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.821877 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.825567 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.827268 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:04.807533 18038 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.000s
I20260709 13:18:04.829432 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.831140 16742 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.760926 18073 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:18:04.791016 17980 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:18:04.832350 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.828140 17973 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.847460 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:04.833140 18072 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.833578 16737 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.817432 18055 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:04.815132 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:04.842257 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.842339 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:04.858994 16742 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.859512 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:04.860862 16742 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.842656 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:04.801760 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:04.846382 16736 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.849152 16667 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.843911 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:04.851468 17980 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.836119 17966 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.848258 17972 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.853407 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:04.857323 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.857936 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:04.807961 18063 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.845938 17952 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.870901 16663 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:04.869668 17973 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.880487 17973 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.864369 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.863802 16493 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.865482 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.845582 18045 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:04.856814 18058 consensus_meta.cc:287] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.000s
I20260709 13:18:04.868156 16742 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:04.863778 18038 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:04.836925 18026 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:18:04.874840 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:04.864360 18023 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.868813 17979 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:18:04.902396 17979 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.902843 16590 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.902850 17979 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:04.902268 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:04.904502 17979 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.889361 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.907609 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.890048 17994 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:04.908344 17994 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.910279 17994 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.868307 17980 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:04.911317 17980 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.912488 17980 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.912590 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:04.890607 18010 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.915717 18010 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.915804 16665 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.916133 18010 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.915937 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:04.916862 17419 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:04.889533 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:04.863138 16740 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:04.899705 16669 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.918509 18038 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.919349 18038 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.921716 17881 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:04.866775 18055 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:04.922144 17881 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:04.887145 18045 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:04.923452 18045 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:04.923025 17881 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.924455 17881 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:04.923034 18053 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:04.925227 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.893769 18058 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:04.928817 16743 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.894004 18025 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:04.900739 18023 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:04.929981 18055 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:04.933054 17973 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.915315 18064 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:04.936030 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:04.937551 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:04.938254 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48803.4us
I20260709 13:18:04.945410 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.956789 16741 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.951656 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:04.946524 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 131607us
I20260709 13:18:04.887967 18063 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:04.965111 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.024056 16669 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:04.978523 16743 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:04.968060 16741 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:04.965571 17929 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:05.050966 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 231792us
I20260709 13:18:05.052035 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.052109 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.963135 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:04.947412 18020 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:04.946878 17972 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:05.033422 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:04.947228 17986 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:04.947926 17973 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:05.053176 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 90116.1us
I20260709 13:18:05.059456 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:04.947052 17967 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:18:05.050315 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:05.042405 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.060479 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:05.059468 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:04.946957 17966 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 13:18:05.084308 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:04.964875 18032 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 13:18:05.043188 18029 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:05.054674 18058 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.039074 17961 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:18:05.056921 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.058462 17952 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.054518 18020 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.039518 18067 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:18:05.057607 18025 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.058670 17972 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:05.089061 18073 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:05.058591 17986 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:05.100701 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:05.110023 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:05.096132 16594 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:05.060582 18072 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:18:05.171844 17415 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:05.158324 17984 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:05.162485 18038 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.164420 17972 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.169473 17950 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:05.166414 18025 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:05.172086 17998 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.194413 16741 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.166916 17986 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.159467 16664 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:05.101416 18029 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.179107 18035 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.182699 17984 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:18:05.191917 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 132554us
I20260709 13:18:05.165704 17992 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.208701 17412 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:05.168570 18073 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:05.217352 18079 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.107s	user 0.008s	sys 0.003s
I20260709 13:18:05.207602 18083 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:05.213801 18035 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.227701 16742 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.216760 17984 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.251441 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:05.249634 18035 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:05.254019 18082 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:18:05.253173 17979 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.256528 17950 tablet_bootstrap.cc:492] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:18:05.225147 18020 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.289582 18020 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.276883 16664 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:05.279592 18080 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:18:05.280508 17992 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.281147 18073 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.283553 18023 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.284554 17959 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.286417 16740 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:05.251012 18083 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.310580 18010 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.269150 18067 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:18:05.322480 17950 ts_tablet_manager.cc:1403] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 1.332s	user 0.029s	sys 0.007s
W20260709 13:18:05.324055 18047 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.710s	user 0.009s	sys 0.002s
I20260709 13:18:05.334239 18010 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:05.257850 18025 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.335760 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:05.341572 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 281147us
I20260709 13:18:05.342374 17415 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:05.337122 17984 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.327342 18045 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.348955 16738 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:05.345952 17973 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.357242 16741 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.364161 17996 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.366444 16559 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:05.370544 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 115934us
I20260709 13:18:05.364653 17996 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.375733 16739 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.377501 16742 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.365170 17984 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.381168 18007 tablet_copy_client.cc:538] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:18:05.380817 17986 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.396425 16594 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.397505 18098 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.397877 17986 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.382118 17412 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.382894 18096 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:05.398600 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 339183us
I20260709 13:18:05.383746 18045 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.389183 18086 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.401466 18098 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.401968 18098 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.389319 18035 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:05.390492 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.404646 18096 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:05.403092 18098 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.405812 18092 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.356294 17950 raft_consensus.cc:359] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.408401 16665 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:05.414705 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.408726 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.372493 17973 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.416978 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.417269 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.415511 16669 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:05.421658 16671 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:05.421785 17415 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:05.425622 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.420348 17929 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.427026 16740 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:05.410292 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.428599 16663 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.431337 16670 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.395494 18078 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:05.432989 16665 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:05.435607 17972 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.437368 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:05.437846 17972 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:05.438427 16589 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:05.438541 17979 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.438414 17996 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.439292 17979 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:05.416329 17950 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:18:05.440263 17979 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.441071 18032 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:05.440985 17950 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.416769 17973 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.442111 17973 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.421375 18092 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.443398 17959 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.444877 18092 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.424584 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.448830 18094 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:05.449324 18094 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.450349 16589 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:05.451154 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50962.9us
I20260709 13:18:05.452651 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:05.452270 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:05.453719 16739 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:05.409787 18086 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:18:05.461222 18022 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.461774 18022 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.453701 18081 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:18:05.462579 17969 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.463076 17969 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:05.464324 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:05.466756 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.464772 17950 ts_tablet_manager.cc:1434] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.000s
I20260709 13:18:05.477303 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:05.464946 17986 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:05.473799 17959 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.519093 17959 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.483208 17975 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:18:05.517496 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:05.521793 17419 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:05.520082 17959 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.507632 16739 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.523213 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:05.463991 17969 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.462766 18022 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.521477 18096 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:05.528945 18047 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:05.450268 18094 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.528987 18054 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:05.534169 18064 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.534819 18007 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:18:05.515030 17986 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:05.515501 18086 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:05.471499 18092 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:18:05.529629 16738 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.576508 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:05.577194 16737 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:05.534924 18099 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:18:05.544725 17952 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:05.548623 17998 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.559948 18045 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.560796 18083 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.561546 16739 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.587011 18045 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.540411 18054 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:05.569422 17986 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.591826 18045 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.588176 18010 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.593408 18010 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.546448 17973 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.475502 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:05.578163 18025 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.580925 18053 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.585410 18047 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:05.585686 17998 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.474889 17929 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:18:05.578459 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:05.579252 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:05.580154 17969 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.624182 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:05.624956 16590 raft_consensus.cc:2368] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
I20260709 13:18:05.626261 16594 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.580356 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:05.627511 17412 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:05.583407 17952 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:05.629230 17952 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.583470 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.631177 16670 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.599213 16743 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:05.474114 17881 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:18:05.636502 17973 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:05.615108 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 144583us
I20260709 13:18:05.639532 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.640121 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.641213 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.513010 18097 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.111s
I20260709 13:18:05.558552 17959 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.619362 16593 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.643121 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:05.650966 16671 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.618502 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:05.612890 18053 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:05.623317 18095 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.623471 18035 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:05.597878 17975 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.631418 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:05.633551 16592 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:05.586272 18086 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.644853 16664 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:05.645081 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.645378 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:05.621735 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.647162 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.617960 17967 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.647531 16666 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:05.647635 17881 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:05.648430 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.649214 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:05.643592 16745 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:05.618813 18038 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:05.655465 16589 raft_consensus.cc:2368] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
I20260709 13:18:05.659929 18029 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.677482 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.618073 18094 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.663372 16591 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:05.663865 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.666729 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:05.679919 16595 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:05.615751 18025 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.667133 16593 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.669265 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:05.664048 18095 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.668653 16587 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:05.672583 16663 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:05.676236 18038 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:05.663793 17975 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.693392 18086 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:05.694145 18010 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:05.683646 18047 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.672884 17967 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.696053 16670 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:05.680063 18035 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:05.685983 18095 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.718538 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:05.695896 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.673975 17998 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.700623 17929 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:18:05.743780 17966 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:05.757335 17881 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.749505 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:05.736327 17973 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
W20260709 13:18:05.773625 18106 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:18:05.691263 18053 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.753877 16664 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:05.795179 17961 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.805018 17975 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.772727 17966 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:05.807436 17984 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:05.813230 18104 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:18:05.816563 17998 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.772563 17881 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.833698 18053 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.836654 17986 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.839892 18025 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.840943 17967 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.843755 18022 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.851948 17980 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.827666 17984 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.816903 17961 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:05.855980 18110 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.080s	user 0.005s	sys 0.000s
I20260709 13:18:05.854965 17986 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:05.906013 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:05.909696 17881 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:05.911877 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:05.856521 17966 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:05.855876 17980 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:05.929462 17984 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:05.922544 17967 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.930816 17967 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.922832 18038 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.940569 16587 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:05.943444 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 379118us
I20260709 13:18:05.951786 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:05.950510 18098 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.953656 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:05.954718 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:05.955415 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:05.956000 17419 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:05.959128 17979 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.959808 17979 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.960342 17979 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.961808 18010 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:05.962445 18010 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:05.962913 18010 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:05.959210 18099 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:05.963976 18099 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:05.966346 18078 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:05.967655 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.968715 16664 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:05.969484 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:05.970602 16663 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:05.973660 17980 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:05.976193 17980 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.978832 18099 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:05.985203 18007 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:18:06.002652 17419 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:05.926739 17961 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:06.006404 18095 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.014447 17969 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.014882 18100 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:06.015285 17969 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.015925 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:06.016247 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-73089175ee9c465197a1a11e304346be received from {username='slave'} at 127.0.0.1:38524
I20260709 13:18:06.016498 17961 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:06.016952 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-73089175ee9c465197a1a11e304346be on tablet 73089175ee9c465197a1a11e304346be with peer ba2df6a668144022a223e504ae571f21
I20260709 13:18:06.018024 17969 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.019608 17967 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.015601 18007 tablet_bootstrap.cc:492] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:18:06.015713 18100 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.021605 18083 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.028589 18083 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.032742 17988 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:18:06.035650 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:06.022722 18023 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.039732 18007 ts_tablet_manager.cc:1403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.505s	user 0.013s	sys 0.024s
I20260709 13:18:06.040432 17992 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:18:06.043687 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.044725 16737 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.045248 18083 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.055733 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.056932 16592 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.057152 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.058367 16663 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:06.038731 18023 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.060878 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.061470 18023 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.062021 16594 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.063174 17415 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:06.063791 16669 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:06.065102 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.065218 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.065892 16669 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.055962 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.068166 18100 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.056954 16589 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.074405 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.075250 18096 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.065945 16595 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.076877 18096 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.077369 18100 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.078213 18100 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.078760 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.084306 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.085223 16737 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.091652 17988 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.095950 17992 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.101747 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.087363 18054 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.116004 17980 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.114511 17988 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.119354 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:06.130424 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:06.133715 16670 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.126114 17980 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.129410 17988 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.137811 18054 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.144670 16737 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:06.146310 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.135350 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.141093 17980 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.146148 16671 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:06.154822 18007 raft_consensus.cc:359] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:06.159824 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.142966 16664 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.170176 18010 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:06.173919 18107 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.467s	user 0.003s	sys 0.001s
I20260709 13:18:06.174368 18053 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:06.175781 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.181794 16670 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:06.161386 18007 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:18:06.160358 18096 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.146435 17992 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.192884 16671 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
W20260709 13:18:06.195396 18105 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.500s	user 0.002s	sys 0.001s
I20260709 13:18:06.203375 16663 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:06.203495 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.204722 16493 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 210) took 1354 ms. Trace:
I20260709 13:18:06.213697 16494 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 232) took 1489 ms. Trace:
I20260709 13:18:06.228436 18095 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.237635 18023 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.183336 18100 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.239908 17952 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.246168 18023 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:06.247135 17969 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:06.247810 17969 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.210364 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.214293 16494 rpcz_store.cc:276] 0709 13:18:04.724655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.724858 (+   203us) service_pool.cc:224] Handling call
0709 13:18:04.753506 (+ 28648us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:18:06.213449 (+1459943us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:05.216185 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:05.280601 (+ 64416us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:05.280631 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:18:05.280650 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:05.280990 (+   340us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:05.281035 (+    45us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:05.281066 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:05.281241 (+   175us) tablet.cc:701] Row locks acquired
0709 13:18:05.281260 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:18:05.281393 (+   133us) write_op.cc:270] Start()
0709 13:18:05.281496 (+   103us) write_op.cc:276] Timestamp: P: 1783603085281349 usec, L: 0
0709 13:18:05.281516 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:18:05.282122 (+   606us) log.cc:844] Serialized 535 byte log entry
0709 13:18:05.561461 (+279339us) op_driver.cc:464] REPLICATION: finished
0709 13:18:05.739395 (+177934us) write_op.cc:301] APPLY: starting
0709 13:18:05.739478 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:05.739570 (+    92us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:05.739698 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:05.739918 (+   220us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:05.740084 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:05.740153 (+    69us) write_op.cc:312] APPLY: finished
0709 13:18:05.740373 (+   220us) log.cc:844] Serialized 23 byte log entry
0709 13:18:05.740576 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:05.740666 (+    90us) write_op.cc:454] Released schema lock
0709 13:18:05.740725 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:05.741106 (+   381us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:05.741155 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":466416,"reported_tablets":1,"spinlock_wait_cycles":24179072,"child_traces":[["op",{"apply.queue_time_us":177712,"apply.run_cpu_time_us":1697,"apply.run_wall_time_us":1692,"num_ops":1,"prepare.queue_time_us":399169,"prepare.run_cpu_time_us":2775,"prepare.run_wall_time_us":66225,"replication_time_us":279833,"thread_start_us":177381,"threads_started":1}]]}
I20260709 13:18:06.261101 17992 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:06.279174 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.279599 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:06.280085 16662 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:06.280387 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 96406.3us
I20260709 13:18:06.205415 16493 rpcz_store.cc:276] 0709 13:18:04.852094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.852271 (+   177us) service_pool.cc:224] Handling call
0709 13:18:06.204412 (+1352141us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:05.282536 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:05.326376 (+ 43840us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:05.331372 (+  4996us) write_op.cc:435] Acquired schema lock
0709 13:18:05.331412 (+    40us) tablet.cc:662] Decoding operations
0709 13:18:05.339342 (+  7930us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:05.339392 (+    50us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:05.339418 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:05.339614 (+   196us) tablet.cc:701] Row locks acquired
0709 13:18:05.339628 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:18:05.339759 (+   131us) write_op.cc:270] Start()
0709 13:18:05.339877 (+   118us) write_op.cc:276] Timestamp: P: 1783603085339717 usec, L: 0
0709 13:18:05.339899 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:18:05.359420 (+ 19521us) log.cc:844] Serialized 535 byte log entry
0709 13:18:05.739700 (+380280us) op_driver.cc:464] REPLICATION: finished
0709 13:18:05.741505 (+  1805us) write_op.cc:301] APPLY: starting
0709 13:18:05.741584 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:05.741716 (+   132us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:05.741839 (+   123us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:05.742050 (+   211us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:05.742256 (+   206us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:05.742323 (+    67us) write_op.cc:312] APPLY: finished
0709 13:18:05.742526 (+   203us) log.cc:844] Serialized 23 byte log entry
0709 13:18:05.742706 (+   180us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:05.742787 (+    81us) write_op.cc:454] Released schema lock
0709 13:18:05.742832 (+    45us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":222248,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":1615,"apply.run_cpu_time_us":1653,"apply.run_wall_time_us":3922,"num_ops":1,"prepare.queue_time_us":392092,"prepare.run_cpu_time_us":3147,"prepare.run_wall_time_us":82475,"replication_time_us":399671,"thread_start_us":73387,"threads_started":1}]]}
I20260709 13:18:06.282133 17374 leader_election.cc:400] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
I20260709 13:18:06.282976 17374 leader_election.cc:403] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:06.255797 18038 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.283139 18095 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:06.283481 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:06.283771 18095 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:06.284183 17409 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:18:06.285161 18126 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.263s	user 0.003s	sys 0.000s
I20260709 13:18:06.284878 18095 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.278242 17984 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.287189 18022 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:06.287626 17984 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.287863 18022 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.289861 18022 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.292018 17992 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.292775 16590 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:06.294281 16559 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.295648 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.296586 18053 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.298537 18083 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.289811 17984 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.302680 17979 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.277741 18007 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:06.311857 18053 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:06.296608 16738 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:06.278043 18010 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:06.327688 18097 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.319008 17988 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.329736 18063 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:06.329659 18097 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:06.330554 18063 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.332916 18063 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.317562 18072 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.334295 18072 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.337167 18072 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.332520 17988 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.339676 17996 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:06.319401 18007 ts_tablet_manager.cc:1434] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.279s	user 0.011s	sys 0.000s
I20260709 13:18:06.332521 18097 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.319854 17980 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.350732 16737 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:06.351555 17374 leader_election.cc:400] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
I20260709 13:18:06.351923 17374 leader_election.cc:403] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:06.352399 17980 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:06.353370 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 169495us
I20260709 13:18:06.316425 17969 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:06.314803 17409 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:06.348978 18098 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.353247 18023 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.370993 18063 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.371569 18063 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.371963 18063 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.379330 16590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee99829f455541f09dac21015b00282e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:18:06.380236 16590 raft_consensus.cc:606] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:18:06.381309 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:06.381600 17966 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.382344 17966 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.385418 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:06.406885 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50586.4us
I20260709 13:18:06.358081 18072 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:18:06.367460 17979 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:06.384296 18038 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:18:06.407840 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.406728 18083 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
W20260709 13:18:06.418085 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66667.7us
I20260709 13:18:06.420471 16667 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:06.392855 18023 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.412750 17966 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:06.391753 18098 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.405750 17984 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:18:06.425571 18073 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.460741 18097 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.461346 18097 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.425860 18089 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.426481 17972 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.427081 18025 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:06.433295 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:06.434983 18093 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:06.451202 16559 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.452210 18081 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:06.451081 17412 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:06.454563 18098 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.465287 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1529) took 1050 ms. Trace:
I20260709 13:18:06.462291 18063 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.472011 16667 rpcz_store.cc:276] 0709 13:18:05.414615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:05.414720 (+   105us) service_pool.cc:224] Handling call
0709 13:18:06.465066 (+1050346us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b5400400e54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:06.465207 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1972508800}
I20260709 13:18:06.475854 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:06.468147 18025 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:06.479588 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 61640.9us
I20260709 13:18:06.474949 17966 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.475242 18093 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:06.487937 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:06.498302 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 115493us
I20260709 13:18:06.499205 17952 raft_consensus.cc:2764] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:06.499291 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:06.481010 18025 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:06.485998 17967 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:06.506007 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.519322 18038 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:06.505453 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.523003 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:06.524235 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 320957us
I20260709 13:18:06.490451 18089 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.519780 18023 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.521121 17967 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:06.510377 18010 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:06.532483 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:06.532063 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.519982 18038 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:06.520522 17979 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:06.530830 18054 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.530957 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:06.552312 16596 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.518623 17966 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:06.561856 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:06.563675 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.564148 18023 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.564298 16588 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:06.564689 18023 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.565152 18010 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.567196 18023 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.567703 18010 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:06.567767 17992 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:06.568569 17992 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.570132 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.570600 17992 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:06.570746 16744 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:06.564955 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.596133 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.560396 18054 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.568416 18010 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.602102 18139 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.604009 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:06.605435 16594 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:06.606220 18140 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:06.613444 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:06.668202 16595 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.569595 18047 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:06.663537 16589 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.665180 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.669502 16667 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.664808 16494 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.574003 17409 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:06.604079 18035 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:06.676378 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 177220us
I20260709 13:18:06.604816 18054 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.682493 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.686268 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.687103 17967 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.688472 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.689293 18029 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.689527 16737 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:06.689849 18029 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.691143 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.692294 18140 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.686439 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 136) took 1976 ms. Trace:
I20260709 13:18:06.600751 17979 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.675689 16494 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 2 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:06.695519 16743 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:06.695189 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:06.720952 16736 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.727422 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.728107 16667 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.684468 18095 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:06.736328 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:06.739724 16744 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.694983 17992 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:06.680292 18035 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:06.746275 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 173171us
I20260709 13:18:06.694842 18140 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:06.717012 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 183837us
I20260709 13:18:06.756630 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:06.757684 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.678241 18010 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.739421 18095 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.760711 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:06.738991 18063 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.755955 16494 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 3 to 4, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:06.773167 18095 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:06.777628 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:06.777839 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:06.778412 16587 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.778738 17952 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:06.780313 18072 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.780740 18072 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:06.781149 18072 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.784232 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:06.785405 16738 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:06.786221 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:06.787036 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:06.788156 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-4951b0fadf5b43e49a71f0bb4fb7a85e received from {username='slave'} at 127.0.0.1:38524
I20260709 13:18:06.788648 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-4951b0fadf5b43e49a71f0bb4fb7a85e on tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with peer ba2df6a668144022a223e504ae571f21
I20260709 13:18:06.788990 18097 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.789665 18097 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:06.790308 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.613389 18093 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.790354 18097 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:06.790923 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.792778 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.798290 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:06.799021 16739 raft_consensus.cc:2368] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:06.786454 17414 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:06.795243 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.801148 17409 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:06.801272 16667 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:06.802659 18089 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.738940 18038 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.803354 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:06.804710 16587 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:06.806137 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.806056 18097 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.790958 16741 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.807816 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.809266 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.810029 18053 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.810853 18053 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.811543 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:06.811686 18053 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.812176 17376 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:06.796030 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.813561 18092 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:06.813953 18092 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:06.814291 18092 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:06.816619 18073 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
I20260709 13:18:06.817431 18073 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.818014 18073 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:06.799947 18138 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:18:06.792037 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.824321 16595 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:06.755402 18035 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:06.788556 18137 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
I20260709 13:18:06.822717 16745 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.826646 17415 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:06.796828 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:06.846915 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 50290.6us
I20260709 13:18:06.695109 18139 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.849161 17992 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.849239 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.851699 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:06.864539 16669 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.810487 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.858196 18073 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.695094 16492 rpcz_store.cc:276] 0709 13:18:04.631244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:04.650368 (+ 19124us) service_pool.cc:224] Handling call
0709 13:18:06.602564 (+1952196us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:04.890358 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:04.890932 (+   574us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:04.890963 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:18:04.890982 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:04.913742 (+ 22760us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:04.913819 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:04.913848 (+    29us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:18:04.914185 (+   337us) tablet.cc:701] Row locks acquired
0709 13:18:04.914203 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:18:04.914362 (+   159us) write_op.cc:270] Start()
0709 13:18:04.914469 (+   107us) write_op.cc:276] Timestamp: P: 1783603084914313 usec, L: 0
0709 13:18:04.914496 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:18:05.023423 (+108927us) log.cc:844] Serialized 1284 byte log entry
0709 13:18:05.400601 (+377178us) op_driver.cc:464] REPLICATION: finished
0709 13:18:05.549177 (+148576us) spinlock_profiling.cc:243] Waited 6.24 ms on lock 0x7b18002b1c80. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327f78c4bc 00007f327cada48b 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f328361aef0 00007f328361b489 00007f328361d71a
0709 13:18:05.818548 (+269371us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:05.820681 (+  2133us) write_op.cc:454] Released schema lock
0709 13:18:06.221600 (+400919us) spinlock_profiling.cc:243] Waited 1.27 ms on lock 0x7b18002b1c80. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1
0709 13:18:06.223401 (+  1801us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b18002b1c80. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:06.223691 (+   290us) write_op.cc:301] APPLY: starting
0709 13:18:06.223838 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:06.224101 (+   263us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:06.224148 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:06.224935 (+   787us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:06.225280 (+   345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:18:06.225363 (+    83us) write_op.cc:312] APPLY: finished
0709 13:18:06.225779 (+   416us) log.cc:844] Serialized 35 byte log entry
0709 13:18:06.275458 (+ 49679us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:06.355340 (+ 79882us) write_op.cc:454] Released schema lock
0709 13:18:06.357940 (+  2600us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:06.420213 (+ 62273us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:06.422152 (+  1939us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":8836,"apply.run_cpu_time_us":3695,"apply.run_wall_time_us":192935,"num_ops":3,"prepare.queue_time_us":141989,"prepare.run_cpu_time_us":23058,"prepare.run_wall_time_us":325568,"raft.queue_time_us":542137,"raft.run_cpu_time_us":51755,"raft.run_wall_time_us":426727,"replication_time_us":485902,"spinlock_wait_cycles":38321536,"thread_start_us":387391,"threads_started":4,"wal-append.queue_time_us":183148}]]}
I20260709 13:18:06.735121 18047 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.875664 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.870630 17992 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:06.884070 17992 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:06.863219 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.892436 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.908522 16739 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.914490 16669 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:06.906034 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:06.921567 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:06.927210 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.879230 18083 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:06.930100 16736 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
W20260709 13:18:06.932513 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 122122us
I20260709 13:18:06.932960 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.891130 18140 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:06.910248 17979 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:06.898908 17980 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.898945 18022 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.934514 16668 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:06.907164 17969 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.940524 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:06.911870 18097 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.903448 17966 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.957273 16744 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:06.895426 16738 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.918963 18147 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:06.964888 16739 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:06.919711 16740 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:06.983263 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:06.971472 17998 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.986778 16736 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:06.894925 17992 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:06.990177 16634 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:06.965549 17967 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:06.969002 18095 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.002794 18141 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:07.003260 18141 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:07.003595 17996 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.003791 17979 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:07.004027 17996 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:07.004511 17996 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.004437 17979 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:06.974725 18097 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:07.006616 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.007059 18148 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:06.972182 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:07.009006 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:06.996191 18140 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:07.009565 16742 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.009869 18140 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.011019 18139 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.012040 18140 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.007403 18097 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.968292 17969 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.015470 18014 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.015873 18097 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.016400 18097 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.016074 18014 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.016736 17969 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.968710 18147 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.020077 16742 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:07.022497 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:06.960558 17980 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:07.035058 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44972.3us
I20260709 13:18:06.989068 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.040048 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:07.038342 16595 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.016844 18097 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.000049 17967 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:07.025408 18038 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:06.968364 18083 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.018450 18014 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:06.966271 18022 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:07.055572 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:07.025811 18073 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.003602 18141 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.027374 18147 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.098702 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:07.046545 18025 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.000s
I20260709 13:18:07.104976 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.106912 16588 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:06.991279 17998 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.018532 18035 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.025250 17881 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:07.111917 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.039498 18047 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.095474 17967 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:07.081343 18053 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:07.100383 17984 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:07.108662 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 125513us
I20260709 13:18:07.031397 18092 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.108188 18148 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:07.098536 18010 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.109915 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.110106 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:07.105877 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:07.110405 16738 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:07.094764 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.126021 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.103574 18157 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:07.109098 18073 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.129989 16739 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.130968 16736 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:07.114107 18143 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:07.108362 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:07.104858 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:07.134855 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:07.137199 16561 leader_election.cc:400] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:07.111009 18025 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:07.137707 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 216199us
I20260709 13:18:07.143247 16561 leader_election.cc:403] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:18:07.139849 18133 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:18:07.140651 17992 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.141554 18054 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.127472 17411 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:07.148178 17998 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.148530 18083 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.149679 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:07.150530 17412 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:07.124560 16737 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:07.152400 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.106135 18038 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.153088 16743 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.154675 18038 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.155076 16559 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:07.155949 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:07.130995 17969 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.123382 18146 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:07.148051 18022 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.119298 17881 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.163579 16710 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:07.179113 16710 leader_election.cc:403] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:18:07.131687 18157 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.123826 18158 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:07.124620 17984 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.167251 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.167917 18092 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.188372 17881 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.167834 18144 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.188817 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.168920 17980 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.169516 16663 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:07.169962 18146 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:07.171677 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46041.9us
I20260709 13:18:07.133941 18148 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:07.172138 18154 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.136468 18053 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.156776 18143 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.175908 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.205739 18146 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.206066 16739 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:07.206373 18146 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.206545 18141 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.177382 18025 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.159796 17969 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.207537 17979 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.207912 17969 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.186646 18038 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:07.209017 18038 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.175948 16738 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:07.156911 16594 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.211190 18038 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.200717 17992 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:07.213899 18053 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.205760 18143 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.184163 18158 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.183177 17966 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:07.215310 17966 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.215736 17966 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:07.216601 18158 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.216789 18148 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.186470 18035 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.217725 16745 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.218502 18095 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.183219 17413 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:07.182343 18157 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:07.188978 17984 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.222761 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.178493 18054 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.189198 18144 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:07.180516 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:07.198269 18159 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.198524 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:07.203979 18154 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:07.206948 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.142620 18089 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:07.168406 18140 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:07.241405 16739 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.197199 17996 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:07.224620 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44185.8us
I20260709 13:18:07.227922 16594 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:07.223609 16738 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
W20260709 13:18:07.246366 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55089.3us
I20260709 13:18:07.189373 18072 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.251333 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:07.235405 18054 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:07.239955 18100 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.011s
I20260709 13:18:07.240794 18144 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.260069 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.260828 16588 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:07.256417 18053 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:07.247754 18147 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.265666 18147 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.245895 18093 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:07.252905 18159 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.289590 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:07.313172 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:07.319650 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 121276us
I20260709 13:18:07.319867 16739 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:07.313369 17415 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:07.321951 18158 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.322570 18158 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:07.322803 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:07.324096 18158 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.326879 16669 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:07.327064 17998 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.327641 17998 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.328074 17998 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.328899 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:07.329653 16669 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:07.335469 18167 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.339954 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.340586 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.342931 16669 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:07.340160 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:07.345077 16670 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:07.352893 16670 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:07.353070 16636 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:07.353492 16636 leader_election.cc:403] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:07.339208 18167 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.357173 18167 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.359512 17998 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.362397 17988 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:07.362514 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.422595 16588 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:07.363423 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.366608 17966 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:07.277662 18100 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.000s	sys 0.012s
I20260709 13:18:07.369868 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:07.235646 18146 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.372304 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:07.414783 18073 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.374127 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.374722 17996 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.375510 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.374660 16668 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:07.257205 18023 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.255204 18054 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.374164 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.388895 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.437125 16742 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:07.437806 16670 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:07.420096 16561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.507s (SENT) [suppressed 40 similar messages]
W20260709 13:18:07.400697 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 47825.1us
I20260709 13:18:07.401890 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:07.404196 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:07.405510 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.439106 16745 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.440122 16662 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:07.408816 18170 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:18:07.401749 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.444851 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.410410 18159 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.403987 18147 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.412748 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:07.413713 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:07.450256 16744 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:07.428685 16669 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.428820 16739 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:18:07.368192 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:07.241041 18154 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.452817 16667 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.454190 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.455344 16742 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:18:07.435030 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 256545us
I20260709 13:18:07.451146 16671 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:07.451155 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83076.5us
I20260709 13:18:07.464371 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:07.464553 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:07.444501 18054 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.444058 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.401621 17967 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.478014 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.449227 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.452476 18053 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.478756 16671 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
W20260709 13:18:07.489440 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 359926us
I20260709 13:18:07.456280 17881 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:07.462337 17996 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.494145 16745 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:07.494937 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.489686 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:07.469833 18140 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:07.503094 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:07.503685 16671 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.483553 18010 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:07.505927 18010 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:07.506022 16663 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:07.532606 18010 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.533260 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:07.533525 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.533595 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.534502 16664 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:07.535351 18140 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.537200 17992 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:07.537263 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.537559 18140 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.538064 16737 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.539085 17996 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.539995 18083 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.540503 18083 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:07.541388 17996 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.542258 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.543008 16737 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:07.543283 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.543792 18161 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:18:07.545202 17998 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.501432 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:07.463083 16740 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:07.543993 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.478361 18147 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:07.586175 18029 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.586651 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.586911 16741 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:07.470757 16590 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:07.514163 16667 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:07.542513 18083 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.508811 16742 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:07.563799 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.605981 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:07.526584 16669 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:07.607518 16739 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
W20260709 13:18:07.597162 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 98813.7us
I20260709 13:18:07.501550 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:07.506500 18014 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:07.608421 18140 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:07.546836 18053 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:07.623106 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 121665us
I20260709 13:18:07.553254 18097 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.534844 17412 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:07.497745 17881 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:07.506644 16587 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.639587 17881 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.634812 17992 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:18:07.641598 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:07.617988 16745 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.632930 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.642675 16739 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.643793 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.644480 16739 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:07.626600 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:07.648564 16739 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:07.648819 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.649634 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.646606 16708 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:07.652678 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:07.634486 16737 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:07.632215 18029 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:07.635084 18140 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:07.636246 17966 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.636633 17411 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:07.636828 18053 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.636763 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:07.637331 18146 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.638381 17996 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.660080 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.695115 16663 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:08.695745 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1552) took 1073 ms. Trace:
I20260709 13:18:08.696117 16663 rpcz_store.cc:276] 0709 13:18:07.622477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.659761 (+ 37284us) service_pool.cc:224] Handling call
0709 13:18:08.695650 (+1035889us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:07.617596 18171 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:07.650282 18177 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:07.656905 16671 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
W20260709 13:18:08.729926 16671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 108) took 1261 ms (client timeout 1509 ms). Trace:
I20260709 13:18:08.730181 16662 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:18:08.730365 16671 rpcz_store.cc:269] 0709 13:18:07.468271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.489890 (+ 21619us) service_pool.cc:224] Handling call
0709 13:18:08.729860 (+1239970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:08.352213 18185 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:08.364369 18187 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:08.731180 18185 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.732014 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1553) took 1068 ms. Trace:
I20260709 13:18:08.732396 16662 rpcz_store.cc:276] 0709 13:18:07.663418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.441902 (+778484us) service_pool.cc:224] Handling call
0709 13:18:08.729928 (+288026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:08.730612 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.731819 (+  1207us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540009ef50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:08.731957 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":661437696}
I20260709 13:18:08.365413 16667 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:08.733378 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1548) took 1181 ms. Trace:
I20260709 13:18:08.733444 18185 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.733748 16667 rpcz_store.cc:276] 0709 13:18:07.551699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.625813 (+ 74114us) service_pool.cc:224] Handling call
0709 13:18:08.733318 (+1107505us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:08.374138 18188 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.375183 18190 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.734526 18188 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.388090 18189 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:08.734653 18190 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.735526 18189 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.736924 18188 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.737212 18190 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.737452 16494 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 211) took 2353 ms. Trace:
I20260709 13:18:08.411545 18169 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.737852 18189 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.738165 18169 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.420588 18195 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.742794 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.743178 18195 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.744820 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:08.745424 16662 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:08.745576 18195 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:08.422513 18196 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.423169 18194 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.423907 18197 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.746397 18196 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.746513 18194 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.746618 18197 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.748787 18196 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.749156 18197 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.748821 18194 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:08.433151 18200 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:08.433692 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.750310 16594 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:08.751380 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.751974 16594 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:08.752882 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.440903 18198 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.753968 16662 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:08.754102 18198 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.755177 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.755787 16662 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:08.756467 18198 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.441468 18202 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:08.457796 18201 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.757238 18202 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.757344 18201 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.759203 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:08.759742 18202 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:08.458539 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.760025 18200 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:08.760198 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:08.760529 16591 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:08.760633 18200 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:08.760998 18169 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.461472 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.761541 18169 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.761914 16590 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:08.761648 18200 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.762163 18169 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.759794 18201 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.763037 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.512136 18207 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.763978 18207 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.764503 18194 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.764923 18194 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.765280 18194 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.527890 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:08.529292 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 1.10914e+06us
I20260709 13:18:08.563772 18205 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.767956 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:08.767791 18205 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.768580 18201 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.769076 18201 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.770087 18205 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:08.560388 18203 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.771147 18203 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.771546 18201 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.772750 18201 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:08.773454 18203 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.773975 18203 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:08.575276 18208 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.775198 18208 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.776085 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 1955) took 1096 ms. Trace:
I20260709 13:18:08.761555 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:08.777282 18208 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.578013 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.777500 16591 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:08.777932 16592 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:08.778486 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:07.673748 16664 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:08.778780 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.779453 16591 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.780457 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1547) took 1229 ms. Trace:
I20260709 13:18:08.751968 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:08.588881 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 65) took 1226 ms. Trace:
I20260709 13:18:08.589022 18216 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.781260 16592 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:08.781416 16589 rpcz_store.cc:276] 0709 13:18:07.362406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.367787 (+  5381us) service_pool.cc:224] Handling call
0709 13:18:08.588043 (+1220256us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b5800010e98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:08.588266 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.588490 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.588524 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:08.588586 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:08.588618 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.588683 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.588801 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13311232}
I20260709 13:18:08.781665 18216 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:08.780880 16664 rpcz_store.cc:276] 0709 13:18:07.551331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.586257 (+ 34926us) service_pool.cc:224] Handling call
0709 13:18:08.780263 (+1194006us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b5400145854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:08.780398 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119222656}
I20260709 13:18:08.783851 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.784287 16589 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:08.784319 18216 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.770624 18205 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:08.785926 16589 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:08.785682 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.786298 16589 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:08.597543 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:08.787281 16593 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:08.603470 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:08.788084 16595 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:08.604501 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:07.664314 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:07.633553 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:08.791520 16744 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:07.663053 16739 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:08.795290 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 1952) took 1147 ms. Trace:
I20260709 13:18:08.795717 16739 rpcz_store.cc:276] 0709 13:18:07.647483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.649422 (+  1939us) service_pool.cc:224] Handling call
0709 13:18:08.795199 (+1145777us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:08.796662 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:08.797281 16739 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:08.370420 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:08.799687 16741 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.326280 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 706) took 1224 ms. Trace:
I20260709 13:18:08.804404 18200 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:08.805022 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:07.675238 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:07.691304 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:08.808274 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 720) took 1400 ms. Trace:
I20260709 13:18:08.808542 16742 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.808686 16740 rpcz_store.cc:276] 0709 13:18:07.408107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.648704 (+240597us) service_pool.cc:224] Handling call
0709 13:18:08.808197 (+1159493us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:08.809835 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:08.810408 16740 raft_consensus.cc:2376] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5. Re-sending same reply.
W20260709 13:18:08.517308 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53304.7us
W20260709 13:18:07.557309 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53003us
I20260709 13:18:08.812844 16591 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:08.817229 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:08.818306 18209 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.622296 18214 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:08.819218 18189 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.819128 18214 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.819761 18189 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.820143 18189 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.820699 18220 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.821214 18220 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.821668 18214 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.374382 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:08.503098 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 361) took 1312 ms. Trace:
I20260709 13:18:08.823488 17415 rpcz_store.cc:276] 0709 13:18:07.190438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.202845 (+ 12407us) service_pool.cc:224] Handling call
0709 13:18:07.203115 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.320856 (+117741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.502664 (+1181808us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400091854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:08.502969 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253075200}
I20260709 13:18:08.823905 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:08.824093 18220 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.663686 17967 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.673691 17969 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:08.830225 17969 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:08.542234 17980 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.283s	user 0.014s	sys 0.049s
W20260709 13:18:08.832608 17980 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.573s	user 0.016s	sys 0.049s
I20260709 13:18:07.647714 17992 raft_consensus.cc:2764] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:18:07.684929 17996 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.836470 16589 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.837404 16589 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:07.661201 18010 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:08.839260 18010 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:08.840148 18010 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.650493 18014 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:07.716183 18023 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:07.546530 17979 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.844632 17979 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:07.681638 18029 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:08.848047 18029 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.849233 17415 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:07.717563 18054 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.852043 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:08.852973 18054 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.408794 18022 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:08.854154 18212 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.854254 18022 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.406697 18072 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.861274 16739 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:07.743321 18047 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.863263 18047 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:08.863813 16744 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:08.864318 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 104) took 1428 ms. Trace:
I20260709 13:18:08.864599 16744 rpcz_store.cc:276] 0709 13:18:07.435697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.633330 (+197633us) service_pool.cc:224] Handling call
0709 13:18:08.864260 (+1230930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:08.865398 18047 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:07.690420 18053 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:08.414765 18073 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.867604 18073 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.869653 18073 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.689052 18097 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.568218 18098 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.891470 18098 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.893652 18098 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.376497 18093 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.894958 18093 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.897262 18093 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.688917 18140 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.506283 18141 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:07.675173 18147 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:07.610096 18157 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.920892 18157 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.664466 18213 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:08.686031 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:08.496291 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:08.697981 18171 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.698494 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:08.698132 18177 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:08.697337 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.705698 18217 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:08.734763 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.386483 18192 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.415911 18191 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:08.740610 16492 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 2, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 2 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:08.737890 16494 rpcz_store.cc:276] 0709 13:18:06.384051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:06.390912 (+  6861us) service_pool.cc:224] Handling call
0709 13:18:08.737359 (+2346447us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:07.015121 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:07.015752 (+   631us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:07.015814 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:18:07.015835 (+    21us) tablet.cc:662] Decoding operations
0709 13:18:07.016488 (+   653us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:07.016576 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:07.016605 (+    29us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:18:07.017047 (+   442us) tablet.cc:701] Row locks acquired
0709 13:18:07.017115 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:18:07.017299 (+   184us) write_op.cc:270] Start()
0709 13:18:07.017432 (+   133us) write_op.cc:276] Timestamp: P: 1783603087017236 usec, L: 0
0709 13:18:07.017453 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:18:07.018195 (+   742us) log.cc:844] Serialized 1289 byte log entry
0709 13:18:07.391146 (+372951us) op_driver.cc:464] REPLICATION: finished
0709 13:18:07.433338 (+ 42192us) write_op.cc:301] APPLY: starting
0709 13:18:07.438562 (+  5224us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:07.456434 (+ 17872us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:07.479767 (+ 23333us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:07.550469 (+ 70702us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:07.550645 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:18:07.550691 (+    46us) write_op.cc:312] APPLY: finished
0709 13:18:07.550911 (+   220us) log.cc:844] Serialized 35 byte log entry
0709 13:18:07.551076 (+   165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:07.551169 (+    93us) write_op.cc:454] Released schema lock
0709 13:18:07.551208 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:07.551541 (+   333us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:07.551573 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19606,"reported_tablets":9,"spinlock_wait_cycles":927104,"child_traces":[["op",{"apply.queue_time_us":7782,"apply.run_cpu_time_us":2350,"apply.run_wall_time_us":128044,"num_ops":3,"prepare.queue_time_us":136603,"prepare.run_cpu_time_us":3690,"prepare.run_wall_time_us":44118,"replication_time_us":373518,"thread_start_us":18162,"threads_started":2}]]}
I20260709 13:18:08.741420 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.936815 16667 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:08.742468 17419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 200) took 1617 ms. Trace:
I20260709 13:18:08.430589 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 1965) took 1408 ms. Trace:
I20260709 13:18:08.756646 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:08.759197 18196 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:08.767215 16669 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:08.979396 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1551) took 1426 ms. Trace:
I20260709 13:18:08.773203 18201 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.939848 16492 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 2 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:08.776435 16736 rpcz_store.cc:276] 0709 13:18:07.682205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.491317 (+809112us) service_pool.cc:224] Handling call
0709 13:18:08.491712 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.751079 (+259367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.751113 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:08.751155 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:08.751182 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.775868 (+ 24686us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400426154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:08.775906 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.776034 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596021120}
I20260709 13:18:08.576332 18193 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.577365 18186 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:08.580502 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:08.983089 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.584312 18211 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:08.789439 18207 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:08.789539 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.777652 18169 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.790692 18194 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.806562 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:08.992257 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 185743us
I20260709 13:18:08.993647 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:08.804462 16743 rpcz_store.cc:276] 0709 13:18:07.101978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.199670 (+ 97692us) service_pool.cc:224] Handling call
0709 13:18:07.202669 (+  2999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:07.219905 (+ 17236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.220022 (+   117us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:07.220275 (+   253us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:07.220402 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.325951 (+1105549us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b540005b754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:08.326011 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.326181 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38266624}
I20260709 13:18:08.790688 16737 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:08.818675 18209 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:08.939898 18192 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.996620 18209 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.998420 18192 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.999204 16664 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:09.000226 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.001062 16664 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:09.001979 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:07.710525 17881 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.826299 18214 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:08.831252 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.070381 16740 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:09.071619 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.072441 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.073021 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.073453 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:08.835196 16666 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:09.074102 16736 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:09.075417 17413 raft_consensus.cc:2368] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
W20260709 13:18:09.075947 16666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 792) took 1920 ms (client timeout 1618 ms). Trace:
I20260709 13:18:08.843364 17409 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
W20260709 13:18:09.076421 17413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 362) took 1714 ms (client timeout 2183 ms). Trace:
I20260709 13:18:09.076759 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 201) took 1616 ms. Trace:
W20260709 13:18:09.076867 17413 rpcz_store.cc:269] 0709 13:18:07.362099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.390000 (+ 27901us) service_pool.cc:224] Handling call
0709 13:18:09.076184 (+1686184us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400465354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:09.076338 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3730640128}
I20260709 13:18:09.077252 17409 rpcz_store.cc:276] 0709 13:18:07.460231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.472590 (+ 12359us) service_pool.cc:224] Handling call
0709 13:18:08.374177 (+901587us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:09.076696 (+702519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86563072}
W20260709 13:18:09.076440 16666 rpcz_store.cc:269] 0709 13:18:07.155818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.159358 (+  3540us) service_pool.cc:224] Handling call
0709 13:18:09.075713 (+1916355us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b540009d154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:09.075878 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3789579520}
I20260709 13:18:09.080467 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.080576 16710 leader_election.cc:400] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:09.080600 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.081094 16710 leader_election.cc:403] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:08.850605 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.081734 18029 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.081696 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.853528 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.082284 18029 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:09.082742 18029 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.083313 18169 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.078555 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.084662 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:08.856799 16670 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:08.859081 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:09.087821 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:08.865670 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.088492 16736 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:09.088948 16744 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:09.089439 17881 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.089761 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.090171 17881 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:08.872437 16742 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:09.091010 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.091140 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 1954) took 1426 ms. Trace:
I20260709 13:18:09.091574 16736 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:09.091537 16742 rpcz_store.cc:276] 0709 13:18:07.664778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.681582 (+ 16804us) service_pool.cc:224] Handling call
0709 13:18:09.091079 (+1409497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:09.092456 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:08.874178 16741 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:09.094894 18029 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.095803 16741 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:08.874442 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 1949) took 1403 ms. Trace:
I20260709 13:18:09.097532 16745 rpcz_store.cc:276] 0709 13:18:07.470788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.645557 (+174769us) service_pool.cc:224] Handling call
0709 13:18:07.660861 (+ 15304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.856310 (+1195449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:08.856335 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:08.856373 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:08.856400 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.874213 (+ 17813us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540005ad54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:08.874262 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.874387 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2748846976}
I20260709 13:18:08.837935 17996 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.879403 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:08.879928 16590 raft_consensus.cc:2368] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3.
I20260709 13:18:08.888432 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:08.897948 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 308) took 1426 ms. Trace:
I20260709 13:18:08.854743 18212 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.857956 18198 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.907605 16589 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:18:09.109349 16589 pending_rounds.cc:85] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 1
I20260709 13:18:09.109596 16589 pending_rounds.cc:107] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:09.109853 16589 raft_consensus.cc:2889] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:09.110219 16742 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:18:08.866698 18053 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.111071 18192 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:09.111524 18212 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:08.924786 16665 raft_consensus.cc:2368] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
W20260709 13:18:09.113358 16665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1542) took 1688 ms (client timeout 1962 ms). Trace:
W20260709 13:18:09.113726 16665 rpcz_store.cc:269] 0709 13:18:07.424663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.424818 (+   155us) service_pool.cc:224] Handling call
0709 13:18:09.113182 (+1688364us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54003defd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:09.113306 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3171395840}
W20260709 13:18:08.926190 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 429970us
I20260709 13:18:09.115345 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:08.926247 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 227997us
I20260709 13:18:08.921809 18213 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:08.890164 18047 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:18:08.939086 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:08.979720 16669 rpcz_store.cc:276] 0709 13:18:07.552596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.667590 (+114994us) service_pool.cc:224] Handling call
0709 13:18:08.979329 (+1311739us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:08.927320 18144 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.560s	user 0.035s	sys 0.067s
I20260709 13:18:08.933714 18141 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:08.937605 17419 rpcz_store.cc:276] 0709 13:18:07.124921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.125175 (+   254us) service_pool.cc:224] Handling call
0709 13:18:07.132040 (+  6865us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:07.146534 (+ 14494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:07.146567 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:07.158146 (+ 11579us) log.cc:844] Serialized 26 byte log entry
0709 13:18:08.561403 (+1403257us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b5400483b20. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fb5b6b8 00007f327fbe492b 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c
0709 13:18:08.561498 (+    95us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:08.561571 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.742147 (+180576us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b5400483854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:08.742202 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:08.742239 (+    37us) raft_consensus.cc:1685] finished
0709 13:18:08.742254 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.742412 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":550,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":152,"spinlock_wait_cycles":95239680,"thread_start_us":130084,"threads_started":1,"wal-append.queue_time_us":143739,"wal-append.run_cpu_time_us":1988,"wal-append.run_wall_time_us":1016456}
I20260709 13:18:08.983356 16668 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:08.981455 16559 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:08.781770 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:08.984884 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:08.939071 18217 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:08.983006 16492 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 2 to 3, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 3 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.934052 18157 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.938916 16663 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:09.004166 16662 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:08.996253 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.122397 17996 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:08.940012 18191 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:09.005913 16737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 1953) took 1344 ms (client timeout 1716 ms). Trace:
I20260709 13:18:09.009907 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:08.989511 18193 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.033326 18194 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.988790 18022 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.988751 18171 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.035209 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.981789 18196 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.005673 18054 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:08.995498 18211 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.033149 18209 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.129714 18169 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:08.979162 16587 rpcz_store.cc:276] 0709 13:18:07.022386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.025218 (+  2832us) service_pool.cc:224] Handling call
0709 13:18:08.430370 (+1405152us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54000514d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:08.430497 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":849514880}
I20260709 13:18:09.038987 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.170090 18217 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.170236 16743 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:09.040432 18177 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.041749 18201 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.047052 18073 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:09.171813 18209 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.172546 17412 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:09.007454 18200 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:09.054128 18189 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.179399 18211 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:09.180142 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 1951) took 1697 ms. Trace:
I20260709 13:18:09.180516 18189 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:08.846791 17979 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:09.088507 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 229490us
I20260709 13:18:09.181216 18200 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:09.182294 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:09.107663 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.183166 16744 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:09.102479 17410 rpcz_store.cc:276] 0709 13:18:07.471784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.483683 (+ 11899us) service_pool.cc:224] Handling call
0709 13:18:08.488003 (+1004320us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:08.488299 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.869824 (+381525us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:08.870159 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:18:08.896678 (+ 26519us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:08.896732 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:08.896785 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:08.897556 (+   771us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:08.897605 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:08.897753 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":588,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":6017,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":3901,"spinlock_wait_cycles":902748288,"thread_start_us":26138,"threads_started":1,"wal-append.queue_time_us":46651}
I20260709 13:18:09.113044 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:08.909175 18014 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.114763 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.116374 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:09.192808 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76495.5us
I20260709 13:18:09.193636 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.194684 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:09.128491 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 346769us
I20260709 13:18:09.128651 16671 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:18:09.128156 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 146777us
I20260709 13:18:09.110662 18029 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.133248 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:09.133464 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 137283us
I20260709 13:18:09.133154 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:09.124497 18144 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.757s	user 0.038s	sys 0.073s
I20260709 13:18:09.127092 18093 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.141002 16708 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:09.247056 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:09.248731 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 107869us
I20260709 13:18:09.250098 16708 leader_election.cc:400] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:09.250362 16708 leader_election.cc:403] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:09.141819 18047 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.128141 18141 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:09.144965 16668 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:09.129642 17967 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.149744 17416 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:09.133719 17996 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.134024 18097 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.134006 18157 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.151111 16596 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
W20260709 13:18:09.138360 16737 rpcz_store.cc:269] 0709 13:18:07.658270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.663997 (+  5727us) service_pool.cc:224] Handling call
0709 13:18:08.995296 (+1331299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:09.155501 18193 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.137681 18205 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.132571 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:08.990681 18186 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.172623 18073 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.180661 16667 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:09.070230 18214 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.087874 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1549) took 1535 ms. Trace:
I20260709 13:18:09.167987 18054 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.186728 16736 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:09.189761 18191 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.180560 16738 rpcz_store.cc:276] 0709 13:18:07.483174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.319481 (+836307us) service_pool.cc:224] Handling call
0709 13:18:08.319907 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:08.928424 (+608517us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:09.088384 (+159960us) log.cc:844] Serialized 14 byte log entry
0709 13:18:09.141425 (+ 53041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:09.141486 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:09.141530 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.179895 (+ 38365us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54003ee9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:09.179943 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.180077 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":133408,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":12893,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":84,"spinlock_wait_cycles":1398990208,"thread_start_us":196313,"threads_started":2,"wal-append.queue_time_us":57322}
I20260709 13:18:09.262477 18205 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.240326 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.168772 18198 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:09.168270 18213 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:09.192077 16665 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:09.182175 18207 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:09.243635 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:09.245982 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.197968 16634 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:09.289498 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45897.6us
I20260709 13:18:09.289898 16743 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:09.290004 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92147.2us
I20260709 13:18:09.290252 16559 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:09.182163 18200 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.283639 16666 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.287465 18205 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.288863 18207 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:09.252573 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.255918 18194 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:09.261076 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.247488 18189 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:09.261312 17416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 307) took 1793 ms (client timeout 1757 ms). Trace:
I20260709 13:18:09.247103 18177 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.242950 18053 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.247839 18192 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.257247 18211 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.257110 18141 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:09.266788 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:09.268163 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:09.267019 16670 rpcz_store.cc:276] 0709 13:18:07.552000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.701340 (+149340us) service_pool.cc:224] Handling call
0709 13:18:07.701640 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:09.087037 (+1385397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.087615 (+   578us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540008be54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:09.087789 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2655941888}
I20260709 13:18:09.273100 17412 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:09.273542 18054 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.258061 18097 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.258442 17996 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.258247 18157 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.239899 16740 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.278396 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.326012 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.327028 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.279673 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:09.283985 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.327983 16662 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:09.328161 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44293.8us
I20260709 13:18:09.328744 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.329324 16662 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:09.330067 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.286239 18186 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.253823 18093 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.291431 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:09.292354 18191 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.294734 18193 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:09.337991 18193 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.251945 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.289017 18187 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:09.258181 18047 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.321998 18192 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.254473 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:09.341236 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86842.9us
I20260709 13:18:09.342208 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.342365 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.343145 16744 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:09.258800 18169 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:18:09.338397 18193 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.349103 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.349866 16741 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:09.310209 16671 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:09.302031 18147 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:09.353411 18147 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:09.353768 18047 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.315248 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.317168 16668 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:09.331830 18097 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.333352 18098 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:09.319360 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 124107us
W20260709 13:18:09.318527 17416 rpcz_store.cc:269] 0709 13:18:07.467766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.497820 (+ 30054us) service_pool.cc:224] Handling call
0709 13:18:08.685737 (+1187917us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:09.261131 (+575394us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54004b6254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:09.261247 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437783680}
I20260709 13:18:09.296774 18196 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.318356 18189 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.333755 16742 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:09.334980 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.335981 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.336939 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.337283 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:09.319926 18177 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.320957 18213 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.295625 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.322585 18194 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:09.347566 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.348271 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.349072 18144 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.352046 18022 raft_consensus.cc:2764] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:09.352589 18203 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:09.352733 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.353853 18147 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:09.354887 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:09.356279 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65544us
I20260709 13:18:09.356801 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.313478 18214 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:09.319265 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 66764us
I20260709 13:18:09.351493 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:09.340870 17967 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:09.408639 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:09.407699 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.365587 16670 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:09.407347 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.410689 16736 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:09.370534 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:09.413676 16744 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.343220 18192 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.342296 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.349359 18187 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:09.439481 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.478071 16668 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:09.459672 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.475251 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.479883 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:09.479357 16741 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:09.481033 16671 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.481496 16739 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.479331 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:09.482950 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.486496 16668 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:09.486713 16743 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:09.407858 18177 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.492234 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.492856 16670 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:09.492623 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.493759 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.494442 16670 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:09.494946 16662 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:09.500557 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.501061 16739 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:09.501426 16745 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:09.417193 18023 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.408226 18144 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.407997 18213 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:09.470947 18073 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:09.439918 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.459040 17969 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.464731 18093 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.499466 18029 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.417701 18098 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:09.458226 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.476454 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.369479 18189 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.408085 18186 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:09.521878 16671 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:09.474597 17996 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:09.419092 18053 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:18:09.436142 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:09.528040 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 157559us
I20260709 13:18:09.573995 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:09.526844 18093 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:18:09.490736 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 159839us
I20260709 13:18:09.559765 16670 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:09.560839 16744 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 3 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:09.473995 18141 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.518587 18144 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.583083 18191 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.584409 18053 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.584972 18053 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.517438 18213 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.523787 17969 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.518457 18023 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.568897 16737 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
W20260709 13:18:09.566646 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 74802us
W20260709 13:18:09.568015 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70907.8us
I20260709 13:18:09.521185 18200 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.524562 18029 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:09.455654 18194 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.575158 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:09.529557 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:09.651829 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 297054us
I20260709 13:18:09.577580 16663 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:09.575807 18093 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:09.580142 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.567724 17410 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:09.566231 17967 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:09.652653 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:09.648895 18203 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:09.656180 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 81089.5us
I20260709 13:18:09.595017 16738 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:09.673885 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.658398 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:09.613142 16664 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:09.587865 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.612672 17996 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.651818 17979 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.677620 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.661047 18093 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:09.608942 16740 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:09.651067 18029 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.570240 18186 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.674155 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:09.608894 18157 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.651199 18010 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.598749 18022 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:18:09.675546 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:09.648947 18200 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:09.633713 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08000828a0 after lost signal to thread 15479
I20260709 13:18:09.685082 16634 leader_election.cc:400] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3.
W20260709 13:18:09.720824 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44433.8us
I20260709 13:18:09.701622 16559 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:09.662367 18238 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:09.597774 18193 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.590052 18211 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:09.590767 18098 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.728773 16669 raft_consensus.cc:2368] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:09.662187 18097 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.668838 18244 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:18:09.661765 18196 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 13:18:09.680003 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 99991.7us
I20260709 13:18:09.682325 18169 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.575299 18187 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.685082 18242 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.704372 18047 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:09.731158 18240 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:09.731933 16665 raft_consensus.cc:2368] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:09.732362 18097 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.586764 18205 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.667599 18054 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.719799 18200 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.754691 16667 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:09.756296 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 78749us
I20260709 13:18:09.681772 18014 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.773710 16736 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:09.686981 18191 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:09.730594 16634 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:09.687981 17979 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.721128 18239 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:09.779430 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:09.781589 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51083.3us
I20260709 13:18:09.782084 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:09.773697 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 98202.3us
I20260709 13:18:09.761030 18201 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.751946 18053 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:09.749254 18241 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:09.772667 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08001db960 after lost signal to thread 16528
I20260709 13:18:09.769635 18047 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:09.783912 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:09.785169 18246 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:09.785207 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.761751 18242 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.724339 18010 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.786021 18022 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.788352 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.760484 18251 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:09.791168 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.748409 18203 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.791915 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.792490 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.792701 18054 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.793082 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.758096 18144 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.795739 18186 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.796118 18157 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.798548 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.775650 18249 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.801931 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.780778 18014 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.805938 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.822054 16592 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:09.807586 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.822741 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.787487 18239 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.789232 18211 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:09.797084 18241 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:09.818245 16666 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.001s
I20260709 13:18:09.802593 18047 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:09.845649 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08002614e0 after lost signal to thread 16620
W20260709 13:18:09.846832 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8200 after lost signal to thread 16695
W20260709 13:18:09.847167 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08003a81c0 after lost signal to thread 16696
W20260709 13:18:09.848220 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08002dcc00 after lost signal to thread 16769
I20260709 13:18:09.791457 18141 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:09.826731 18196 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:18:09.849045 18141 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.849409 18141 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.849758 18141 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.813834 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.811594 18022 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.851035 18022 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.811851 18251 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:09.804910 18053 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:09.808233 18242 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.774971 17967 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.819916 16590 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:09.821347 16587 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.784505 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:09.820174 18023 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.827548 18241 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.803469 18207 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.838578 18189 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.828441 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.826318 18014 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.830008 18010 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:09.826453 18249 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.830621 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.832341 17979 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.834038 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.839430 18193 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.840574 18205 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.815941 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 813) took 1229 ms. Trace:
I20260709 13:18:09.825089 18239 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.930514 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:09.918963 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.932440 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:09.933630 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.842680 16666 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:09.848505 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec5e0 after lost signal to thread 16770
W20260709 13:18:09.809857 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 88093.4us
I20260709 13:18:09.854604 18251 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.854754 18053 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:09.841964 18147 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:09.801088 18201 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:09.826063 18246 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:09.882408 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.882652 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:09.884775 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.885381 16591 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.891743 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:09.860791 17969 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.825701 18192 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.878728 18238 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:09.897950 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 824) took 1139 ms. Trace:
I20260709 13:18:09.897864 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.865754 18097 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:09.902765 18144 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.913545 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:09.913686 18029 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:09.920079 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:09.979803 18053 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.011584 16668 raft_consensus.cc:2376] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3. Re-sending same reply.
I20260709 13:18:09.924045 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:10.012624 16739 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:09.924511 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:10.013955 16744 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:09.920090 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:09.923544 16736 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:09.935989 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.016566 16741 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:09.936666 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.017495 16745 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.937489 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.941759 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:09.942468 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:10.024083 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 240025us
I20260709 13:18:09.943411 18189 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:09.945178 18249 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:09.945603 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:09.898465 18198 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:09.931594 16664 rpcz_store.cc:276] 0709 13:18:08.586216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.034459 (+448243us) service_pool.cc:224] Handling call
0709 13:18:09.808164 (+773705us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b540000ffd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:09.814117 (+  5953us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062606592}
I20260709 13:18:09.948835 18177 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:09.884913 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 110238us
W20260709 13:18:09.964735 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 142067us
I20260709 13:18:09.981436 16670 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:18:09.990609 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 98921.9us
I20260709 13:18:09.996179 16592 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:09.996001 16671 rpcz_store.cc:276] 0709 13:18:08.766996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.571591 (+804595us) service_pool.cc:224] Handling call
0709 13:18:09.571895 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:09.572200 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:09.572232 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:09.572764 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 13:18:09.887861 (+315097us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:09.887900 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:09.887962 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:09.897216 (+  9254us) raft_consensus.cc:1685] finished
0709 13:18:09.897242 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:09.897868 (+   626us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":30596,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":21376,"thread_start_us":309611,"threads_started":1,"wal-append.queue_time_us":322834}
I20260709 13:18:09.998317 16662 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:09.941900 18093 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.918031 18047 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:09.992403 18246 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:10.025153 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:10.055015 18201 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:10.026139 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.055800 16596 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.056568 18047 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:10.057605 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 217) took 2654 ms. Trace:
I20260709 13:18:10.059232 18093 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.059414 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 798) took 1477 ms. Trace:
I20260709 13:18:10.060405 18189 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.059826 16493 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:10.061336 17414 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:10.061053 18189 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:09.979991 18054 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:18:10.063681 18189 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.027912 16737 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.065565 18177 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.066411 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:10.067914 18147 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:10.027486 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08003038e0 after lost signal to thread 17446
I20260709 13:18:09.994827 18213 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.052495 16669 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:09.997021 18235 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:10.077257 16741 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:10.055272 18246 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:10.027941 16591 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.033722 16740 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.072556 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.081270 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:10.072468 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 108370us
I20260709 13:18:10.073941 16745 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
W20260709 13:18:10.075425 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50337.7us
I20260709 13:18:10.003036 18097 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:10.086305 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 156449us
W20260709 13:18:10.075523 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08003713e0 after lost signal to thread 17447
I20260709 13:18:10.088178 16591 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:10.058000 16492 rpcz_store.cc:276] 0709 13:18:07.403522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:07.405974 (+  2452us) service_pool.cc:224] Handling call
0709 13:18:10.057550 (+2651576us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:09.303045 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:09.338095 (+ 35050us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:09.338130 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:18:09.338152 (+    22us) tablet.cc:662] Decoding operations
0709 13:18:09.465161 (+127009us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:09.465221 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:09.465256 (+    35us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:18:09.465573 (+   317us) tablet.cc:701] Row locks acquired
0709 13:18:09.465590 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:18:09.465738 (+   148us) write_op.cc:270] Start()
0709 13:18:09.465863 (+   125us) write_op.cc:276] Timestamp: P: 1783603089465692 usec, L: 0
0709 13:18:09.465886 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:18:09.466528 (+   642us) log.cc:844] Serialized 1289 byte log entry
0709 13:18:09.806299 (+339771us) op_driver.cc:464] REPLICATION: finished
0709 13:18:09.866706 (+ 60407us) write_op.cc:301] APPLY: starting
0709 13:18:09.866825 (+   119us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:09.867059 (+   234us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:09.867278 (+   219us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:09.867891 (+   613us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:09.868132 (+   241us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:18:09.868197 (+    65us) write_op.cc:312] APPLY: finished
0709 13:18:09.868483 (+   286us) log.cc:844] Serialized 35 byte log entry
0709 13:18:09.868705 (+   222us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:09.868817 (+   112us) write_op.cc:454] Released schema lock
0709 13:18:09.868867 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:09.869259 (+   392us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:09.869286 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":183603,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":60101,"apply.run_cpu_time_us":2529,"apply.run_wall_time_us":2535,"num_ops":3,"prepare.queue_time_us":121281,"prepare.run_cpu_time_us":47188,"prepare.run_wall_time_us":290457,"raft.queue_time_us":74611,"raft.run_cpu_time_us":6040,"raft.run_wall_time_us":48417,"replication_time_us":340240,"thread_start_us":289121,"threads_started":4,"wal-append.queue_time_us":201618}]]}
I20260709 13:18:10.087702 18189 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:10.080338 18191 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:10.080317 18235 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.110283 16737 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.087909 18249 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:10.114466 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.115639 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:10.119134 16668 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.116653 16559 leader_election.cc:400] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.119702 16559 leader_election.cc:403] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.086128 18238 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.171473 16592 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.155078 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.138180 16559 leader_election.cc:400] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.228888 16559 leader_election.cc:403] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.123479 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.134157 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:10.230283 16671 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.122159 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:10.120712 16669 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:10.231069 16662 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:10.235019 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 96990.6us
I20260709 13:18:10.099781 18189 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:10.059868 16666 rpcz_store.cc:276] 0709 13:18:08.581736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:09.082801 (+501065us) service_pool.cc:224] Handling call
0709 13:18:10.059257 (+976456us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54003df4d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:10.059356 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443018624}
I20260709 13:18:10.099857 18200 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.155877 17979 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
I20260709 13:18:10.251525 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.236963 18191 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.252746 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.237051 18169 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:10.132808 16587 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.254419 18169 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.126744 18241 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:10.249481 18251 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.256501 18169 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:10.253772 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.256881 18241 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.257468 18189 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:10.142612 18141 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.258755 16594 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:10.258999 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:10.259824 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.260268 18141 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:10.260892 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.261554 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.261952 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.257660 18251 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.110980 18235 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.258301 16588 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.106675 18073 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:10.262243 16594 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.263993 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.280707 16671 raft_consensus.cc:2368] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.280907 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.267784 16668 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:10.281754 18073 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:10.282038 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.282622 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.282785 18073 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.283604 16668 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.284539 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.285364 16668 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.286197 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.098954 18014 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.258458 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.296427 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.297150 16669 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.141350 18192 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.304996 16559 leader_election.cc:400] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.305377 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:10.305966 16559 leader_election.cc:403] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.281766 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.306982 16671 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.136656 18029 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.126731 18242 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:10.124300 18262 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.171s
I20260709 13:18:10.265915 18239 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.287261 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.316085 18239 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:10.316375 16664 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:10.316864 16589 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.317624 16591 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.235410 18265 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.319412 16740 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.266045 18250 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:10.319442 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.320199 16588 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.320212 18250 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.321807 18014 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.322592 18250 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:10.262574 16593 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.324045 18200 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.325039 18203 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.132855 16663 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:10.313027 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:10.329771 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:10.330569 16666 raft_consensus.cc:2368] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:10.135793 18207 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.331187 18169 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:10.124099 18097 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:10.146099 18213 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:10.334321 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.335091 16592 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.335520 16588 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
W20260709 13:18:10.119856 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53539.5us
I20260709 13:18:10.309463 16671 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:10.311998 18141 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:10.310071 18246 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.341605 18141 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:10.312423 16590 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:10.313035 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.313411 18242 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.336222 16593 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:10.348927 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.349550 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.350211 16593 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:10.126675 18022 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:10.343535 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 242689us
I20260709 13:18:10.344012 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.351169 18198 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:18:10.379158 18264 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.116s	user 0.003s	sys 0.000s
I20260709 13:18:10.384213 18189 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.390031 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:10.390437 16669 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:10.393659 16708 leader_election.cc:400] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:10.394330 16708 leader_election.cc:403] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.395992 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.395763 18200 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:10.399345 16667 raft_consensus.cc:2368] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
W20260709 13:18:10.396605 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 281969us
I20260709 13:18:10.400244 18249 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:10.400573 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.400964 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:10.401562 18249 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.401803 16667 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:10.404489 18244 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.403800 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.407641 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.397054 16559 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.398537 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.410573 16593 raft_consensus.cc:2368] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:10.410751 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.411851 16591 raft_consensus.cc:2368] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:10.412914 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.413838 16559 leader_election.cc:400] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
I20260709 13:18:10.414391 16559 leader_election.cc:403] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:18:10.415231 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.413733 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:10.426524 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.428572 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.431631 16668 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.437136 16493 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 137) took 1845 ms. Trace:
I20260709 13:18:10.437181 16591 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:10.440255 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.440587 16662 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.444022 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.389798 18169 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.386587 18235 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:10.440600 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 336435us
I20260709 13:18:10.445925 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.445858 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:10.454355 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.454959 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.423259 18246 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:10.342626 18141 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:10.450889 16591 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:10.393183 18242 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:18:10.405009 18244 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.559273 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:10.475248 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61600.5us
I20260709 13:18:10.504035 18198 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.570255 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.571206 16596 raft_consensus.cc:2368] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:10.458060 16666 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:10.565704 18022 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.437759 16493 rpcz_store.cc:276] 0709 13:18:08.591745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:08.598132 (+  6387us) service_pool.cc:224] Handling call
0709 13:18:10.437062 (+1838930us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:10.087640 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:10.120568 (+ 32928us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:10.120605 (+    37us) write_op.cc:435] Acquired schema lock
0709 13:18:10.120617 (+    12us) tablet.cc:662] Decoding operations
0709 13:18:10.120987 (+   370us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:10.121043 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:10.121068 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:18:10.121247 (+   179us) tablet.cc:701] Row locks acquired
0709 13:18:10.121260 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:18:10.121417 (+   157us) write_op.cc:270] Start()
0709 13:18:10.121518 (+   101us) write_op.cc:276] Timestamp: P: 1783603090121373 usec, L: 0
0709 13:18:10.121533 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 13:18:10.122122 (+   589us) log.cc:844] Serialized 530 byte log entry
0709 13:18:10.289559 (+167437us) op_driver.cc:464] REPLICATION: finished
0709 13:18:10.290005 (+   446us) write_op.cc:301] APPLY: starting
0709 13:18:10.290085 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:10.290185 (+   100us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:10.290278 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:10.290518 (+   240us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:10.290684 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:18:10.290734 (+    50us) write_op.cc:312] APPLY: finished
0709 13:18:10.290941 (+   207us) log.cc:844] Serialized 23 byte log entry
0709 13:18:10.291112 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:10.291187 (+    75us) write_op.cc:454] Released schema lock
0709 13:18:10.291225 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:10.291548 (+   323us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:10.291575 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":314422,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":244,"apply.run_cpu_time_us":1498,"apply.run_wall_time_us":1498,"num_ops":1,"prepare.queue_time_us":1394,"prepare.run_cpu_time_us":2550,"prepare.run_wall_time_us":39776,"replication_time_us":167884}]]}
I20260709 13:18:10.464874 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.492174 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.502193 16665 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:18:10.574477 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 109726us
I20260709 13:18:10.509155 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.474668 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.577113 16592 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.474095 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.472105 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.503633 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.578087 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:10.462422 18177 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 13:18:10.498705 18235 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.578366 16589 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.502079 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.472662 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.586048 16745 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.578639 16742 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.455705 16667 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.501435 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.435618 18207 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:10.416254 18200 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:10.424626 18192 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:10.580633 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 106597us
I20260709 13:18:10.581647 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.582010 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:10.586453 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.607378 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.537225 16668 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:10.493510 18268 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:18:10.473215 18023 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:10.603711 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.617218 16596 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:10.617551 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:10.604808 16667 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:10.447688 18203 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.605103 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.623183 16589 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:10.547592 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.472877 18213 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:10.631719 16591 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:10.464277 18169 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:10.471807 18240 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:10.602407 16737 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.582957 18238 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:10.564811 16590 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:10.587018 17969 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:10.616006 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.587162 18022 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:10.620201 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.660982 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.621187 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.440280 18053 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.618911 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.629814 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:10.496646 18189 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:10.669060 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:10.669862 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.613663 16745 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:10.675010 18203 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:10.676357 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:10.676950 18277 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:18:10.677196 16587 raft_consensus.cc:2368] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:10.632855 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.635288 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.678102 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.646057 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.678789 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.631225 18192 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:10.588495 18157 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:18:10.622948 18023 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:10.631580 18235 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:10.640050 18177 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:10.616074 18239 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:18:10.633574 18200 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:10.663726 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44905.2us
I20260709 13:18:10.663870 16742 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:10.665169 16737 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:10.665302 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.682113 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.684002 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.636231 18213 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.789439 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:10.887328 16588 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:10.658077 18250 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
I20260709 13:18:10.788949 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:10.849825 16595 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:10.874569 16591 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:10.660076 18022 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:10.875792 16592 raft_consensus.cc:2376] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3. Re-sending same reply.
I20260709 13:18:10.774535 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.668849 18238 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:10.790445 16668 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.885967 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.687852 16741 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.895735 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.896205 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:10.867300 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 198299us
I20260709 13:18:10.679661 18249 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:10.905706 16744 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:10.877080 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.938478 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:10.720117 18278 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:18:10.755492 18246 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:10.787639 18279 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:18:10.913141 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.723878 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:10.709092 18192 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:10.926679 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:10.990249 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.039729 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:10.989996 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.221130 16596 raft_consensus.cc:2368] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:11.221313 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:10.959666 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:11.099241 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 211010us
I20260709 13:18:11.100693 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.018805 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.219180 16745 raft_consensus.cc:2368] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
W20260709 13:18:11.225327 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 312251us
I20260709 13:18:11.170838 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.218919 16743 raft_consensus.cc:2368] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:11.220465 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.234438 16742 raft_consensus.cc:2368] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:10.811792 18253 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:10.916584 16739 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:11.239661 16739 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:11.222579 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:10.722213 18053 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.241264 16739 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:10.971660 17374 leader_election.cc:336] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.509s (SENT)
I20260709 13:18:10.897275 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.227525 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:10.687419 18214 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:10.807595 18240 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:11.224629 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.226402 16634 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:10.729635 18169 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:11.019807 18277 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:18:10.938297 18073 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:11.266076 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 43600.5us
I20260709 13:18:11.129187 16740 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:11.267953 16592 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:10.889890 18023 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.909020 18207 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:11.269712 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:11.269994 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.236950 18157 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:11.285835 16737 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:10.937988 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:11.270780 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.271091 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.271333 16634 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:11.264065 16745 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:10.933215 18141 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.338s
I20260709 13:18:11.249197 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:11.164588 18213 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.163470 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:10.712179 18193 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.035315 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:11.243459 18253 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:10.892477 17979 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.356s
I20260709 13:18:10.990681 18244 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:11.279892 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.308590 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.308712 16594 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.287501 18187 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.020306 17969 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.244807 16738 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:10.727676 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:11.263463 16596 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:11.287487 18157 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:11.265261 18250 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.287371 16591 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.287739 16588 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:11.286653 16592 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:11.289949 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.201154 18238 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.231928 18022 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:11.322834 17420 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Advancing to term 2
I20260709 13:18:11.302034 16744 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:11.299791 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.305154 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:11.306195 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:11.308749 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.324920 16737 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:11.325119 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:11.237102 18177 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.232407 18246 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.312183 16742 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:11.250382 18249 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.317087 16743 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.249068 18240 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.293996 17413 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:11.280524 18073 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.317960 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.299041 18287 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:11.309747 18198 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:11.327075 16592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "446649301a1c45808dccfebfadfed54c"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:18:11.403199 16592 raft_consensus.cc:606] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:11.402468 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96384.3us
I20260709 13:18:11.307968 18214 raft_consensus.cc:2764] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:18:11.328586 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 640375us
I20260709 13:18:11.394533 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.325996 18265 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:11.383153 16594 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:11.359619 18147 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:11.307688 18205 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 13:18:11.413403 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 89255.8us
I20260709 13:18:11.319731 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 841) took 1074 ms. Trace:
I20260709 13:18:11.310616 17410 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:11.419940 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:11.418332 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.447911 16737 raft_consensus.cc:2368] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
I20260709 13:18:11.448776 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.449046 18073 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:11.449510 16737 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:11.350795 16591 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:11.319543 18193 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.102s
I20260709 13:18:11.450649 16737 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:11.322652 18244 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.316756 18157 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.427664 18246 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.427531 18265 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:11.395972 18239 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:11.456173 18244 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.456755 18239 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.457369 18244 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:11.360140 18187 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.458616 18241 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:11.459079 18241 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:11.459247 18187 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.459847 16741 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:11.459955 18241 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.393220 18253 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.461566 18249 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.430276 18198 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.423389 18286 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:11.461617 16741 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:11.463323 18286 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.464510 18286 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.355297 18177 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:11.452502 18297 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.467124 18177 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.467720 18297 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.468191 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.435849 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.444998 16743 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:11.303525 18200 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:11.355273 18250 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:11.483777 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 212502us
W20260709 13:18:11.483959 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 517327us
I20260709 13:18:11.486701 16745 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:11.488338 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 265843us
I20260709 13:18:11.525381 16737 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:11.536779 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.537634 16736 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:11.537472 17374 leader_election.cc:336] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.521s (SENT)
I20260709 13:18:11.537535 16671 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:11.521615 16590 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:11.475248 18198 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:11.521881 18253 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:11.582666 16589 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:11.632169 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 235543us
I20260709 13:18:11.585427 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:11.634981 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.635555 16591 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.598415 18287 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.628341 18073 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.655910 16591 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:11.659554 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.445945 16664 rpcz_store.cc:276] 0709 13:18:10.245333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.317225 (+ 71892us) service_pool.cc:224] Handling call
0709 13:18:10.317542 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.405322 (+ 87780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.405358 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.405914 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:18:10.476534 (+ 70620us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:10.485962 (+  9428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:10.509320 (+ 23358us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:11.307333 (+798013us) raft_consensus.cc:1685] finished
0709 13:18:11.308794 (+  1461us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.316042 (+  7248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87360,"raft.queue_time_us":90853,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":39254,"threads_started":1,"wal-append.queue_time_us":181304}
I20260709 13:18:11.643096 17414 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:11.650117 18239 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:11.664551 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:11.672127 16591 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:11.663987 18309 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.673372 18309 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.674710 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.675647 18309 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:11.676036 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.676863 16592 raft_consensus.cc:2410] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:11.677608 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.650378 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:11.679898 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.680259 18198 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.681035 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.681923 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.678818 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.543818 17969 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:11.683744 16592 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:11.696236 17420 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 2 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 13:18:11.702926 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 286276us
I20260709 13:18:11.662279 18310 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:11.716857 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.721124 16743 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:11.713807 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:11.691810 18306 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:11.722702 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 184057us
I20260709 13:18:11.699193 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:11.738583 16743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:39096
I20260709 13:18:11.710054 18246 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:18:11.742522 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:11.745528 16743 raft_consensus.cc:606] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:11.747251 16743 raft_consensus.cc:614] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:11.650926 18073 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.747277 17420 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:11.752696 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:11.753420 16670 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:11.714732 18308 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:11.775180 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 248450us
I20260709 13:18:11.775681 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:11.758673 17420 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:11.732514 18192 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:11.660632 18297 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.785256 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:11.749198 18300 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.794298 16671 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:11.799041 16590 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:11.766099 17979 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:11.765972 18309 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:11.766307 18318 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:11.818684 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 72891.5us
I20260709 13:18:11.832607 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:11.749477 18177 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.848733 17413 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:11.800400 18293 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.850386 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:11.797156 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:11.852871 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:11.788409 16738 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:11.834803 18318 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:11.734594 18310 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.858417 18318 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.804731 18250 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:11.808015 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 862) took 1013 ms. Trace:
I20260709 13:18:11.826469 17979 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.796787 18205 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 13:18:11.863839 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:11.866743 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:11.867657 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 92337.4us
I20260709 13:18:11.868494 16744 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:11.873873 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 84080.5us
I20260709 13:18:11.745195 18306 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.797709 18192 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.875015 18244 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:11.883983 18244 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.884343 18244 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:11.885264 18296 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:11.885756 18296 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.859429 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:11.851483 18309 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:11.887334 18316 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:18:11.887694 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:11.887892 18296 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:11.888938 18309 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:11.889139 18322 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:18:11.813472 18141 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:18:11.875188 18323 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:11.853763 18214 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:18:11.900966 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:11.805233 18300 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.890168 18246 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:11.896597 17414 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:11.892194 18187 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.766515 18287 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.801016 18308 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:11.894546 18311 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:18:11.908175 18329 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:18:11.922829 18177 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:11.903807 18323 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:11.924705 18310 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.906072 17979 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.939211 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.909155 18246 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:11.861763 18293 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:11.978717 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:11.965555 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:11.965739 18306 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.933286 18333 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.966480 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.968174 18300 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:11.898156 18304 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:18:11.979379 16736 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:11.935434 18309 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:11.982923 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:11.955822 18240 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.000s
I20260709 13:18:12.021081 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.024366 16736 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:11.962793 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.025282 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:11.953328 18187 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:12.021081 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 56884.6us
I20260709 13:18:12.025503 16742 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:11.972239 18311 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:18:12.028425 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 227546us
I20260709 13:18:12.014966 18246 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.032682 16708 leader_election.cc:400] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:11.862479 16663 rpcz_store.cc:276] 0709 13:18:10.791254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.935938 (+144684us) service_pool.cc:224] Handling call
0709 13:18:10.936288 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.090422 (+154134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.095012 (+  4590us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:11.096542 (+  1530us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:11.121525 (+ 24983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.787263 (+665738us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b540001b154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:11.795155 (+  7892us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:11.804471 (+  9316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119053696}
I20260709 13:18:12.041874 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:11.950073 18198 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:18:12.043454 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:12.045001 16561 leader_election.cc:400] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:12.025288 18306 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.048804 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.053396 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.049407 16561 leader_election.cc:403] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:18:12.055219 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.056349 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.056597 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.057967 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.022689 17979 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:12.024868 18333 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:12.060990 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:12.040380 18309 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:12.060117 16738 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:12.041265 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 152165us
I20260709 13:18:12.076653 18147 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:12.051851 18235 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:12.059396 18141 raft_consensus.cc:2764] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:18:12.059141 18300 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:12.086912 16561 leader_election.cc:400] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:12.061518 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 118) took 1560 ms. Trace:
I20260709 13:18:12.061115 18296 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.046568 18250 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:18:12.088037 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:12.095824 16742 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:12.028119 18240 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.061611 18338 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.063927 18213 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:18:12.061240 18169 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.039912 18318 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.096930 18073 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:12.077430 17979 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.124270 18311 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.126032 16670 raft_consensus.cc:606] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.138697 16561 leader_election.cc:400] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3.
I20260709 13:18:12.130543 16666 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:12.133908 16741 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:12.079674 18304 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:18:12.133737 17979 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:12.141227 18306 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:18:12.147890 18213 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.135353 18338 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:12.149089 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:12.129920 18023 consensus_meta.cc:287] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.838s	user 0.007s	sys 0.005s
W20260709 13:18:12.149932 18341 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:18:12.130072 18318 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:12.150746 18022 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.782s	user 0.011s	sys 0.000s
W20260709 13:18:12.150947 18327 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:18:12.122282 18300 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:12.151454 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.151934 18293 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:12.138947 18147 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:18:12.124928 18323 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.128099 18346 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:18:12.153646 18265 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:12.118371 16740 rpcz_store.cc:276] 0709 13:18:10.499586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.507777 (+  8191us) service_pool.cc:224] Handling call
0709 13:18:10.517412 (+  9635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.278256 (+760844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.031168 (+752912us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b5400427554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:12.059969 (+ 28801us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153644288}
I20260709 13:18:12.124723 18187 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.208760 18022 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.130460 16664 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:12.155426 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.202399 18023 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.180928 18310 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:18:12.235014 16737 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:12.156718 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:12.236173 16662 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:12.236670 18335 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:12.130614 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:12.131726 18157 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:18:12.158026 18244 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.119594 18198 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.142794 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:12.148989 18295 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:12.159288 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:12.308745 18295 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:12.309010 16738 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:12.179443 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.187239 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:12.101694 18235 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.190351 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:12.193557 18207 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.315054 18207 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:12.315884 18207 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.316824 18207 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.317415 18246 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.194329 18205 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.318641 16743 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:12.318799 18205 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:12.319320 18294 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.796s	user 0.005s	sys 0.000s
I20260709 13:18:12.319649 18205 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.180603 18340 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.320936 18246 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:12.196035 18332 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:18:12.321584 18235 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:12.196257 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:12.324695 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 856) took 1649 ms. Trace:
I20260709 13:18:12.200992 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:12.208137 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.212428 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:12.325567 16742 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:12.214752 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.331658 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.210733 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:12.331956 18300 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:12.332527 16736 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:12.208021 18309 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:12.225407 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:12.191432 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:12.196317 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:12.130290 18240 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:12.202750 17414 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
W20260709 13:18:12.238914 18339 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:18:12.258538 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:12.352125 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:12.341856 16590 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.353017 18300 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:12.353261 16594 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:12.353427 18300 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:12.353878 18300 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.335745 18306 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:12.355839 18306 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.357774 18240 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:12.235849 18250 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.219260 18241 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.308073 18244 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:12.178061 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:12.311817 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 169087us
I20260709 13:18:12.314625 16738 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:12.318958 16669 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:12.304659 18198 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:12.164790 18192 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:12.325062 16667 rpcz_store.cc:276] 0709 13:18:10.675076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.675229 (+   153us) service_pool.cc:224] Handling call
0709 13:18:10.675542 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:10.889310 (+213768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:10.889338 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:10.902736 (+ 13398us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.178841 (+276105us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:11.184054 (+  5213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.184166 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.197065 (+1012899us) spinlock_profiling.cc:243] Waited 8.78 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:12.322152 (+125087us) raft_consensus.cc:1685] finished
0709 13:18:12.322171 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.324635 (+  2464us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32623,"raft.queue_time_us":36991,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":72,"spinlock_wait_cycles":20204928,"thread_start_us":324911,"threads_started":2,"wal-append.queue_time_us":275107}
I20260709 13:18:12.181519 18338 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:12.334404 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:12.257803 18157 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:12.336262 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 123928us
I20260709 13:18:12.338256 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:12.339594 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:12.385301 17416 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.235612 18265 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.341501 16742 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:12.385340 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:12.343123 18326 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 13:18:12.360556 18305 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.564s	user 0.000s	sys 0.004s
I20260709 13:18:12.203241 18333 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.310267 18308 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:12.363814 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 307532us
W20260709 13:18:12.369084 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 266901us
I20260709 13:18:12.375313 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2018) took 1895 ms. Trace:
I20260709 13:18:12.341543 18193 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.363373 18250 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:12.390321 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.397977 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:12.405138 16708 leader_election.cc:400] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:12.406176 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.397761 16589 rpcz_store.cc:276] 0709 13:18:10.480078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.237796 (+757718us) service_pool.cc:224] Handling call
0709 13:18:11.271743 (+ 33947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:11.597696 (+325953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.375109 (+777413us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54000104d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:12.375258 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713835392}
I20260709 13:18:12.408527 18297 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.409194 18297 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.410475 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:12.412166 18246 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.412729 17416 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:12.413844 18235 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:12.382761 18286 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:12.415606 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.386925 18336 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:12.417984 16741 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:12.396816 18240 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:12.427958 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.427368 18192 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:12.420614 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.398226 18198 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.431500 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.423774 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:12.438205 18246 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.438293 18189 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:12.417560 18235 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:12.446991 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.445956 18189 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:12.447989 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.448194 18189 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.340313 18295 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.418736 18286 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:12.398553 18177 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:18:12.453545 18246 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.455510 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:12.455670 18297 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.454515 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.429087 18244 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:12.428793 18157 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:12.449692 18235 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.473874 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:12.461598 16741 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:12.476070 18293 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.498175 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 756) took 1201 ms. Trace:
I20260709 13:18:12.498746 16739 rpcz_store.cc:276] 0709 13:18:11.297071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.297853 (+   782us) service_pool.cc:224] Handling call
0709 13:18:11.298143 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:11.298339 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:11.298363 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:11.306492 (+  8129us) log.cc:844] Serialized 26 byte log entry
0709 13:18:11.605767 (+299275us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:11.605804 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:11.605880 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:12.358490 (+752610us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:12.497832 (+139342us) raft_consensus.cc:1685] finished
0709 13:18:12.497853 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:12.498096 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5168384,"thread_start_us":287992,"threads_started":1,"wal-append.queue_time_us":399323}
I20260709 13:18:12.454217 18306 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:12.503687 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.504392 16745 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:12.508536 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.512137 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.494529 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:12.481900 18295 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:12.471680 18198 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.521502 16588 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:12.476070 18244 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:12.571437 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:12.516872 18351 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:12.573184 16739 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:18:12.570431 18177 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:12.496115 18193 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:12.571602 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59568.1us
I20260709 13:18:12.453725 18308 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
W20260709 13:18:12.589759 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.583s (SENT) [suppressed 36 similar messages]
I20260709 13:18:12.579972 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:12.591248 16666 raft_consensus.cc:606] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.577857 17969 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:12.582924 18097 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:12.584654 16590 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:12.579665 18353 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:12.578583 16739 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 2
I20260709 13:18:12.589264 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:12.592425 16741 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:12.594838 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.567972 18297 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.578603 18244 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:12.458592 18338 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.578176 18246 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:12.616904 16708 leader_election.cc:400] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:12.619482 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.633870 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:18:12.476117 18286 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.636281 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:12.638406 16745 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:12.611049 18205 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:12.577695 18351 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:12.613371 16739 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:18:12.627487 18293 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:12.591022 18177 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.593760 18193 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.611533 18097 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:12.635779 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:12.605515 17969 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:12.665283 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:12.666325 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49545.1us
W20260709 13:18:13.026441 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:12.650383 18338 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:12.688967 16745 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:12.691670 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 358757us
I20260709 13:18:12.658120 18265 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:12.637123 18353 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:12.720667 16739 raft_consensus.cc:2917] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:18:12.799925 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 852) took 2325 ms. Trace:
I20260709 13:18:12.715692 18351 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:12.915750 18147 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.002s	sys 0.012s
W20260709 13:18:12.916369 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 349216us
W20260709 13:18:12.945703 18323 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.017s	sys 0.026s
W20260709 13:18:12.946271 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 356485us
W20260709 13:18:13.065403 16594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.025s	sys 0.038s
I20260709 13:18:13.025676 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 1984) took 1088 ms. Trace:
I20260709 13:18:13.026681 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:13.038758 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.067452 16743 rpcz_store.cc:276] 0709 13:18:11.937260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.937384 (+   124us) service_pool.cc:224] Handling call
0709 13:18:12.723728 (+786344us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54003d4354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:13.025612 (+301884us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":696350080}
I20260709 13:18:13.050050 16739 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.050575 16662 rpcz_store.cc:276] 0709 13:18:10.395563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.535729 (+140166us) service_pool.cc:224] Handling call
0709 13:18:10.536182 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.236604 (+1700422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.720850 (+484246us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540009cc54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:12.721047 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3909145856}
W20260709 13:18:13.034986 18239 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.012s	sys 0.030s
I20260709 13:18:12.665674 18308 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:13.068475 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:13.037997 18253 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.431s	user 0.002s	sys 0.019s
W20260709 13:18:13.067432 16594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.025s	sys 0.038s
I20260709 13:18:13.080360 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:13.082650 16744 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:13.032971 18340 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:13.070657 18147 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.003s	sys 0.012s
W20260709 13:18:13.071002 18323 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.018s	sys 0.027s
I20260709 13:18:13.111523 16743 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:13.125458 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45207us
W20260709 13:18:13.090543 18239 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.012s	sys 0.031s
I20260709 13:18:13.082788 18293 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:13.147815 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86058us
I20260709 13:18:13.089783 18308 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:13.152518 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:13.156957 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.110837 18246 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:13.111675 18249 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:13.168083 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.083596 18244 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.200373 16743 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:18:13.165699 18310 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:13.118525 18297 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 13:18:13.155221 18187 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:13.114156 18340 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.209884 16561 leader_election.cc:400] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
I20260709 13:18:13.157594 18157 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
W20260709 13:18:13.228920 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 121016us
I20260709 13:18:13.174317 18323 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:18:13.186618 18265 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:18:13.234397 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:13.235694 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.187304 18249 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.251653 17416 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:18:13.561360 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 325726us
I20260709 13:18:13.562016 17416 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Advancing to term 3
I20260709 13:18:13.563359 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:13.564111 18249 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.564325 18362 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.564845 18324 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:13.565181 18362 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:13.254817 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:13.566668 18324 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:13.567543 17412 raft_consensus.cc:2410] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:13.567885 18324 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:13.568379 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:13.255534 18235 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.005s
I20260709 13:18:13.568830 18324 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.569198 18235 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.569540 18386 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:13.571238 18386 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.571416 18235 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:13.257369 18189 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.801s	user 0.016s	sys 0.000s
I20260709 13:18:13.224830 18310 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:13.572353 18310 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.572676 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:13.568728 17412 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.574985 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:13.254885 18193 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:13.259913 18295 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.686s	user 0.005s	sys 0.009s
I20260709 13:18:13.242985 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:13.650045 18382 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:13.650231 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 407337us
I20260709 13:18:13.232113 18365 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:13.250226 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:13.247510 18187 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:13.293802 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:13.295578 18360 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:18:13.296322 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:13.652900 18187 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.296422 17414 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.297793 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:13.299029 18364 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:18:13.299963 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:13.300287 18244 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.655009 18193 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.302263 18022 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.309437 18381 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:18:13.288267 18207 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.971s	user 0.009s	sys 0.012s
W20260709 13:18:13.289932 18241 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.923s	user 0.018s	sys 0.000s
I20260709 13:18:13.656067 18381 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:13.290280 18300 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.936s	user 0.017s	sys 0.006s
I20260709 13:18:13.655819 18193 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:13.656692 18300 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.656945 18381 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.658072 18193 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.658175 18381 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.658824 18300 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:13.232621 18308 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.659649 18308 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.265753 18147 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.663136 18147 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.664243 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:13.310954 18177 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.336351 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:13.665252 18147 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.317664 18097 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.336647 18317 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
W20260709 13:18:13.665992 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 329694us
I20260709 13:18:13.333019 17969 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.333619 18192 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:18:13.667186 18300 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:13.663072 16744 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.355420 18351 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.371408 18023 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.375044 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:13.376474 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:13.668905 16736 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:13.391862 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2047) took 1028 ms. Trace:
I20260709 13:18:13.418522 18380 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:13.669328 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2003) took 1235 ms. Trace:
I20260709 13:18:13.357635 18340 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:13.424597 18250 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:13.428684 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 272277us
I20260709 13:18:13.669601 18380 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.438113 18383 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:13.670300 18250 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.670692 18383 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.671959 18380 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.425251 18306 raft_consensus.cc:993] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:18:13.672695 18306 raft_consensus.cc:1081] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:18:13.672833 18383 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:13.669755 16736 rpcz_store.cc:276] 0709 13:18:12.433829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.374670 (+940841us) service_pool.cc:224] Handling call
0709 13:18:13.669273 (+294603us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:13.501987 18358 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.350s	user 0.003s	sys 0.000s
I20260709 13:18:13.542583 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 1993) took 1545 ms. Trace:
W20260709 13:18:13.446981 18359 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.405s	user 0.004s	sys 0.000s
I20260709 13:18:13.670250 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:13.551023 18238 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.675961 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 1994) took 1677 ms. Trace:
I20260709 13:18:13.675853 18238 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.676343 18097 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:13.676908 18097 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:13.677901 18238 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.552224 18326 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:13.577612 18362 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:13.677834 18097 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.573725 18363 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:13.258412 18205 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.001s
I20260709 13:18:13.578626 18386 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:13.572316 18189 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:13.556672 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:13.586225 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.604285 17416 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:13.584386 18310 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:13.613933 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.617195 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2005) took 1182 ms. Trace:
W20260709 13:18:13.619675 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.626569 18295 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:13.628966 16593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1568) took 3111 ms (client timeout 1917 ms). Trace:
I20260709 13:18:13.669641 18351 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:13.621758 18235 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:13.631196 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 131) took 1219 ms. Trace:
I20260709 13:18:13.632160 18297 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.633872 16587 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.669358 16596 rpcz_store.cc:276] 0709 13:18:12.363173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.385638 (+ 22465us) service_pool.cc:224] Handling call
0709 13:18:12.385920 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.606839 (+220919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.606913 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.617393 (+ 10480us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.948238 (+330845us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:12.948295 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.948394 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.391544 (+443150us) raft_consensus.cc:1685] finished
0709 13:18:13.391569 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.391781 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":242894,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6580,"thread_start_us":495821,"threads_started":2,"wal-append.queue_time_us":584493}
I20260709 13:18:13.695600 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1605) took 2019 ms. Trace:
I20260709 13:18:13.629290 17412 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:18:13.651975 17410 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:13.655129 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 385) took 1765 ms. Trace:
I20260709 13:18:13.635972 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:13.337488 18353 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:13.676724 18177 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:13.678781 18362 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:13.678746 18326 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.579756 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:13.667145 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:13.679466 18363 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:13.696228 16587 rpcz_store.cc:276] 0709 13:18:11.675874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.681933 (+  6059us) service_pool.cc:224] Handling call
0709 13:18:13.633632 (+1951699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.693135 (+ 59503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.695302 (+  2167us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b54002369d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:13.695514 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4487774592}
I20260709 13:18:13.714452 18363 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:13.680298 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.676378 16740 rpcz_store.cc:276] 0709 13:18:11.998263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.239287 (+241024us) service_pool.cc:224] Handling call
0709 13:18:12.239645 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.240005 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.240034 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.240471 (+   437us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.605703 (+365232us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:13.108737 (+503034us) log.cc:844] Serialized 14 byte log entry
0709 13:18:13.156839 (+ 48102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.165282 (+  8443us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.501384 (+336102us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:13.675634 (+174250us) raft_consensus.cc:1685] finished
0709 13:18:13.675730 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.675897 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":608661,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":2984,"raft.queue_time_us":56544,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":3044,"spinlock_wait_cycles":17036672,"thread_start_us":786945,"threads_started":2,"wal-append.queue_time_us":816901}
I20260709 13:18:13.721828 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.722458 16740 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:13.687043 16739 rpcz_store.cc:276] 0709 13:18:12.434406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.500313 (+ 65907us) service_pool.cc:224] Handling call
0709 13:18:12.500632 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.091446 (+590814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.616958 (+525512us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b54002b93d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:13.617137 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165751168}
I20260709 13:18:13.723716 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:13.686102 18346 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:18:13.679950 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2035) took 1489 ms. Trace:
I20260709 13:18:13.685824 18309 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:13.748965 18309 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.723883 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.725620 16591 rpcz_store.cc:276] 0709 13:18:12.190664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.190988 (+   324us) service_pool.cc:224] Handling call
0709 13:18:12.191368 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.217283 (+ 25915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.217335 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.218007 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.596044 (+378037us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:12.596088 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.596162 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.446286 (+850124us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:13.679540 (+233254us) raft_consensus.cc:1685] finished
0709 13:18:13.679562 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.679890 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40080,"raft.queue_time_us":11787,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":19562,"spinlock_wait_cycles":257674112,"thread_start_us":357146,"threads_started":1,"wal-append.queue_time_us":462587}
I20260709 13:18:13.692979 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.694355 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:13.754765 16745 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 2: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:13.755447 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:13.755757 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.756743 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:13.694240 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.758561 16736 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:13.761586 16736 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:18:13.762075 16736 pending_rounds.cc:85] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 1
I20260709 13:18:13.762373 16736 pending_rounds.cc:107] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:13.690368 16662 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:13.690963 16667 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:13.697747 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 70866.8us
I20260709 13:18:13.694478 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:13.704905 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2046) took 1360 ms. Trace:
I20260709 13:18:13.689893 16666 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:13.685948 17409 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:13.687386 16593 rpcz_store.cc:269] 0709 13:18:10.517348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:10.660742 (+143394us) service_pool.cc:224] Handling call
0709 13:18:13.628692 (+2967950us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54002369d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:13.628868 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4743337728}
I20260709 13:18:13.687870 18351 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.692662 18235 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.842453 18351 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.844596 18235 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.844849 18351 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:13.845731 18351 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:13.707989 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 128343us
I20260709 13:18:13.707113 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:13.674474 16737 rpcz_store.cc:276] 0709 13:18:11.996777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.235883 (+239106us) service_pool.cc:224] Handling call
0709 13:18:12.236861 (+   978us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.237262 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.237298 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.237945 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.589889 (+351944us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:13.027967 (+438078us) log.cc:844] Serialized 14 byte log entry
0709 13:18:13.028388 (+   421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.028515 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.542269 (+513754us) raft_consensus.cc:1685] finished
0709 13:18:13.542296 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.542499 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":617823,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":669,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":15765,"thread_start_us":758979,"threads_started":2,"wal-append.queue_time_us":960471}
I20260709 13:18:13.847313 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:13.848168 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.849476 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.849736 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.692667 16738 rpcz_store.cc:276] 0709 13:18:12.411894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.630185 (+1218291us) service_pool.cc:224] Handling call
0709 13:18:13.630562 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.630768 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:13.630813 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:13.630892 (+    79us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:13.630930 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.630997 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.631134 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:13.851588 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.852314 16738 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:13.853286 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2033) took 1672 ms. Trace:
I20260709 13:18:13.729645 16740 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.856457 18378 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:13.856974 18378 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:13.857385 18378 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.665755 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:13.705394 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.752595 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.860555 16739 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:13.860991 18253 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:13.861740 16739 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.861505 18253 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.775005 16595 rpcz_store.cc:276] 0709 13:18:12.344657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.381572 (+ 36915us) service_pool.cc:224] Handling call
0709 13:18:12.381845 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.382030 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.382055 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.388980 (+  6925us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.827269 (+438289us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:12.827333 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:12.827432 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.704693 (+877261us) raft_consensus.cc:1685] finished
0709 13:18:13.704714 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.704848 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":404780,"threads_started":1,"wal-append.queue_time_us":906609}
I20260709 13:18:13.762665 16736 raft_consensus.cc:2889] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.783347 16662 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:13.783191 18386 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:18:13.784998 16666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "446649301a1c45808dccfebfadfed54c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36074
I20260709 13:18:13.808029 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:13.865792 16743 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:13.866518 18386 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.708904 18326 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:13.867344 18326 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:13.867770 18326 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:13.868669 18326 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.870090 18326 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.847337 16664 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:13.871620 18300 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:13.854111 18239 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:18:13.873351 18239 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:13.873689 18239 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:13.875948 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:13.876722 18386 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:13.877472 18386 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:13.877784 18386 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:13.878152 18386 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.824635 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2009) took 1388 ms. Trace:
I20260709 13:18:13.828706 17409 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:13.853677 16592 rpcz_store.cc:276] 0709 13:18:12.181366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.189774 (+  8408us) service_pool.cc:224] Handling call
0709 13:18:12.190099 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:12.414737 (+224638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.414769 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:12.414810 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:12.414841 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.853025 (+1438184us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540010d954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:13.853074 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.853228 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516118784}
I20260709 13:18:13.697170 17420 rpcz_store.cc:276] 0709 13:18:11.891478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:11.892183 (+   705us) service_pool.cc:224] Handling call
0709 13:18:11.892508 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.025088 (+132580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.025115 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.025551 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:18:12.375731 (+350180us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b1800129d00. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:18:13.024976 (+649245us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:13.025015 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.025084 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.654816 (+629732us) raft_consensus.cc:1685] finished
0709 13:18:13.654840 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.655072 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":129471,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":2789,"spinlock_wait_cycles":8259456,"thread_start_us":478188,"threads_started":2,"wal-append.queue_time_us":1050848}
I20260709 13:18:13.878943 16741 rpcz_store.cc:276] 0709 13:18:12.436462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.617854 (+181392us) service_pool.cc:224] Handling call
0709 13:18:12.618299 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.629137 (+ 10838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.629187 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.644208 (+ 15021us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.138974 (+494766us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:13.139016 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.139088 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.824352 (+685264us) raft_consensus.cc:1685] finished
0709 13:18:13.824377 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.824569 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2454,"raft.queue_time_us":129,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":17791,"thread_start_us":417794,"threads_started":1,"wal-append.queue_time_us":679264}
W20260709 13:18:13.835047 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55506.9us
I20260709 13:18:13.881662 18253 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:13.859970 18235 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:13.882987 18235 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.697881 17412 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 2
I20260709 13:18:13.884239 17412 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:18:13.884497 17412 raft_consensus.cc:2917] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:18:13.884902 17412 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:13.885003 18235 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:13.874490 18239 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.886473 18239 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:13.886983 16739 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:13.888052 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:13.889292 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.889870 16739 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:13.691920 18365 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.201s
I20260709 13:18:13.850595 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.892930 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.893572 16737 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:13.810815 18309 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:13.892235 16636 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Timed out: RequestConsensusVote RPC to 127.15.29.4:41455 timed out after 1.757s (SENT)
W20260709 13:18:13.899873 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 234188us
I20260709 13:18:13.900689 18286 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.901377 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:13.902443 18300 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.899125 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:13.903889 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.905118 18235 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.902561 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:13.709096 18238 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:13.905969 18238 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:13.906333 18238 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:13.908782 16664 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.909535 16662 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:13.682775 18310 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:13.913373 16737 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:13.914177 18386 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:13.916138 16634 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.969135 18386 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:13.969791 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53743.1us
I20260709 13:18:13.970791 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.972378 18323 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:18:13.972860 18240 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:13.916688 18193 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:18:13.979382 17374 connection.cc:256] RPC call timeout handler was delayed by -1.0185938740000893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24630000000000002s for a response
W20260709 13:18:13.980125 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.612s (SENT)
I20260709 13:18:13.980571 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:13.980851 18382 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.981427 18382 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.983549 18382 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:13.905210 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:13.984830 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79686.4us
I20260709 13:18:13.985356 18286 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:13.905544 18309 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:13.916162 16669 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:13.679054 18022 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.249s
I20260709 13:18:13.986963 16743 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:13.930598 16668 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:13.987913 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.988624 16665 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.988741 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:13.989065 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 136) took 1394 ms. Trace:
I20260709 13:18:13.989332 16671 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:13.890663 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:13.936400 18324 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:13.937434 18250 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:13.885494 16740 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:13.683140 18200 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.328s
I20260709 13:18:13.996017 18365 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:14.011708 18324 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:14.011755 18365 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:13.942041 16593 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.013619 18365 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.014891 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:14.015157 16737 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:14.016141 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:13.711341 18353 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:13.942451 16662 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.019138 18193 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:13.997215 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2044) took 1695 ms. Trace:
I20260709 13:18:14.020542 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:13.682420 18308 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.339s
I20260709 13:18:13.990409 17996 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:13.944367 16592 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.023034 16662 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:13.949080 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2016) took 1255 ms. Trace:
I20260709 13:18:13.949524 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.023918 16589 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:14.024858 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.024456 18324 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:13.951563 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:13.953045 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.026024 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:14.026551 18324 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.027695 18193 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:14.028282 18193 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:14.028687 18193 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:14.034186 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.035516 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.014731 18365 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.026082 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:13.956596 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.707471 18362 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.038582 16742 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.039834 16742 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:14.039891 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.040681 16665 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:14.040977 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.041581 16662 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:14.041601 16742 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:13.679183 18244 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.286s
I20260709 13:18:14.045322 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:13.965307 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.687356 18295 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.047626 18295 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:14.023680 16744 rpcz_store.cc:276] 0709 13:18:12.694063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.116777 (+422714us) service_pool.cc:224] Handling call
0709 13:18:13.117089 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.668105 (+551016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.948836 (+280731us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54003ee9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:13.949015 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1255228288}
I20260709 13:18:14.049424 16744 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:14.050521 16744 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:14.043439 16742 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:13.697122 18317 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.051739 16742 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:14.052079 16742 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:13.943547 16664 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:13.965984 18333 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:13.965664 18346 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:13.985352 16663 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:13.681255 18189 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.019217 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.019520 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:13.989506 16668 rpcz_store.cc:276] 0709 13:18:12.595079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.595302 (+   223us) service_pool.cc:224] Handling call
0709 13:18:12.595578 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:13.988171 (+1392593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.988846 (+   675us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001f7054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:13.989015 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3069420416}
I20260709 13:18:13.697836 18340 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:13.999620 16671 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:14.025576 16589 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:13.951035 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.027699 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.052700 18362 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.080528 16587 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.081614 18362 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.082983 16666 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:14.083314 18300 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:14.083413 16666 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:13.967152 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:13.962687 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:14.090065 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109969us
I20260709 13:18:14.062927 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:14.092022 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.111044 16670 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:14.110980 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85012.4us
W20260709 13:18:14.104097 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 138864us
I20260709 13:18:14.068645 16588 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:18:14.067307 16596 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:14.113421 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:13.989334 16743 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:14.062484 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 75709.1us
I20260709 13:18:13.924396 18351 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:14.127805 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:13.936919 18239 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:13.938273 18238 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.090303 16666 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.116827 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:14.174753 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:14.175495 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.020282 16594 rpcz_store.cc:276] 0709 13:18:12.302034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:12.354332 (+ 52298us) service_pool.cc:224] Handling call
0709 13:18:12.354660 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:12.373352 (+ 18692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:12.373379 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:12.384236 (+ 10857us) log.cc:844] Serialized 26 byte log entry
0709 13:18:13.102223 (+717987us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:13.102260 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:13.102335 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:13.996907 (+894572us) raft_consensus.cc:1685] finished
0709 13:18:13.996936 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:13.997143 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16173,"raft.queue_time_us":44608,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":14629,"thread_start_us":684281,"threads_started":1,"wal-append.queue_time_us":950879}
I20260709 13:18:14.123447 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.176162 16588 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 1.
I20260709 13:18:14.176700 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.177070 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.178004 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:14.091459 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.565s (SENT)
W20260709 13:18:14.179183 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55846.9us
I20260709 13:18:14.179674 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:14.121237 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:14.180075 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.119984 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.133911 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:14.182698 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.190666 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.183037 16593 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:14.185650 18097 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.187386 18383 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:18:14.188143 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.192981 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:14.193872 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.054824 16742 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:14.188843 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44670.1us
I20260709 13:18:14.191792 16670 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:14.194998 16742 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:14.196301 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 78992.9us
I20260709 13:18:14.196549 16593 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:14.192844 18381 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:14.203022 16587 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:14.136231 18205 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:14.135999 18333 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:14.204891 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 83696.1us
W20260709 13:18:14.206254 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.206534 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.056802 18200 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:14.060364 18336 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:14.092157 18189 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.197100 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:14.092295 18363 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.092335 18198 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.053414 17996 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:14.092379 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:14.092543 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:14.220973 16591 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:14.222148 16739 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:18:14.262306 16739 pending_rounds.cc:85] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 1
I20260709 13:18:14.262566 16739 pending_rounds.cc:107] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:18:14.262960 16739 raft_consensus.cc:2889] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.114352 18286 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.092118 18270 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.066210 18340 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:14.228081 18306 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.226121 18253 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:18:14.231446 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.234840 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.195748 18147 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.243889 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:13.679731 18023 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.540s
W20260709 13:18:14.247786 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.211403 18371 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:14.077140 18193 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:14.254437 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 57469.7us
I20260709 13:18:14.189706 18343 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:18:13.692932 18297 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.189256 18308 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:18:14.188612 18239 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.202531 18097 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.065685 18378 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.251210 18177 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:14.189374 18382 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:14.204330 18207 consensus_queue.cc:1175] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 2 status { last_received { term: 2 index: 3 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 3 } }
I20260709 13:18:14.245975 18200 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:14.251157 18336 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:14.270309 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.266979 18286 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.255134 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:14.258370 18240 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:14.257302 18250 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.277938 16634 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:14.275390 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 56465.2us
I20260709 13:18:14.243352 18338 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:18:14.290496 18371 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:14.262710 18333 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:14.339351 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 61519.7us
I20260709 13:18:14.340062 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:14.282665 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.264923 18411 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.254351 18235 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:18:14.288270 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:14.250905 18205 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:14.260177 18189 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.259380 18363 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.360363 18417 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:18:14.360852 16588 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:18:14.361259 16588 pending_rounds.cc:85] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 4
I20260709 13:18:14.361485 16588 pending_rounds.cc:107] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:18:14.361707 16588 raft_consensus.cc:2889] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:14.361826 18363 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.354944 18411 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:14.363581 18187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.006s	sys 0.024s
W20260709 13:18:14.363956 18187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.006s	sys 0.024s
I20260709 13:18:14.364339 18411 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.352041 18383 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:18:14.365711 18383 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:14.365764 18378 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.327813 16595 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:14.286707 18241 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.263084 18353 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:18:14.367476 18297 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:14.368976 18097 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.369452 18351 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:14.289798 18022 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:14.370749 18193 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.371129 18022 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:14.371572 18022 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.332502 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:14.374392 18407 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 13:18:14.331303 18240 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:14.271497 18306 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.344446 18340 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.374468 16744 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:14.344861 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62258.4us
I20260709 13:18:14.347990 18362 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:14.303995 18382 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.378856 18371 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.379688 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.380275 16744 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:14.349701 18333 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:14.292881 17969 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:18:14.330989 16590 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:14.366132 18383 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:14.394644 16596 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:14.271255 18270 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.351907 18310 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.414290 16744 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.414358 16589 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:18:14.423089 16594 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:14.351802 18323 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:14.413635 18414 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:18:14.335029 17996 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.375100 18240 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.431872 16744 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:18:14.423959 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 295345us
I20260709 13:18:14.435232 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:14.437529 18378 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.437991 18378 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.438385 18378 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.439035 16743 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:18:14.440284 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 105068us
I20260709 13:18:14.442408 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:14.441807 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58504.4us
W20260709 13:18:14.445302 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8462290009999833s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:18:14.422876 18097 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.344103 18250 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:14.423353 18265 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.458103 16592 raft_consensus.cc:2410] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:14.456120 18205 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.456525 18097 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:14.461429 18097 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.430182 18323 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:14.445686 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6641056359999311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1583s for a response
I20260709 13:18:14.446973 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:14.467581 16708 connection.cc:256] RPC call timeout handler was delayed by -1.62630563599987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 13:18:14.468029 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4124647389996881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
I20260709 13:18:14.464300 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.469465 16595 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:14.451515 18324 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:14.442052 18382 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:14.452175 18425 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:14.453469 16666 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.453598 18270 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.476567 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.445566 16634 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.470157 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:14.442754 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.486239 18187 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.486680 18187 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:14.487207 18187 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.488129 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:14.488546 16740 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:14.489874 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.469147 18323 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.495051 16587 raft_consensus.cc:2410] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:14.493054 18382 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.498075 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.400437 18336 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.501979 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.475257 18324 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.503430 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.505965 16737 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:14.490775 16740 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:14.500635 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:14.506585 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:14.503485 16741 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:14.501902 18241 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:18:14.496991 18425 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.469815 18265 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.510367 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:14.511130 16737 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:14.525753 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 237551us
I20260709 13:18:14.525895 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:14.526600 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:14.517539 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.511940 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.534409 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:14.527662 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:14.536676 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:14.539285 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 447810us
I20260709 13:18:14.538018 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:14.546610 18340 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.547233 18340 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:14.547693 18340 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.548650 18265 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.474028 18383 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.549588 18383 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:14.550983 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:14.552678 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:14.554188 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:14.507704 18193 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:18:14.532330 18431 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:14.522742 18380 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:14.518339 18385 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:14.509486 18324 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:14.517887 18239 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.543828 18189 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.599126 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.600520 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.601822 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.508044 18253 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:18:14.551337 18147 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.544857 18386 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:14.674469 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.557803 18382 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:14.606516 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.606312 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.677716 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:14.607545 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.678498 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:14.544628 18338 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.131s
W20260709 13:18:14.684686 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.685760 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.708350 16587 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:14.709474 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.602937 18380 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:14.716634 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.690423 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.595782 18418 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.594067 18295 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:14.697065 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:14.684420 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.698365 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.544041 18343 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.699952 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:14.596261 18177 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.539564 18205 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 13:18:14.603907 18427 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:14.717835 16596 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:14.723991 16589 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:14.736758 16589 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:14.737269 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.737927 16596 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:14.737536 16589 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.738580 16589 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.726702 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.679944 18385 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:14.681088 18147 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:14.693686 18386 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.733675 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:14.697852 18382 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:14.550226 18378 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.736011 18177 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:14.698084 16663 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:14.736990 18343 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:14.715023 18351 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:18:14.741139 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024f2e0 after lost signal to thread 15479
W20260709 13:18:14.774897 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024aba0 after lost signal to thread 16528
I20260709 13:18:14.738816 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:14.776355 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080019ec60 after lost signal to thread 16620
I20260709 13:18:14.777290 16596 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:14.777510 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024f8a0 after lost signal to thread 16695
W20260709 13:18:14.756979 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.760702 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:14.779989 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003c4700 after lost signal to thread 16769
I20260709 13:18:14.768143 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.750576 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:14.781589 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7c60 after lost signal to thread 17446
W20260709 13:18:14.771273 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 231227us
W20260709 13:18:14.787003 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.463s (SENT)
I20260709 13:18:14.739789 16594 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:14.754232 18177 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:14.804221 18433 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:14.755753 18425 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.807376 18425 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:14.807473 18306 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.752960 18386 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:14.801204 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:14.828799 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.829653 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:14.830833 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.832945 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.834209 17411 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:14.747432 18147 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.835532 18286 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.837142 17419 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 2 LEARNER]: Advancing to term 3
I20260709 13:18:14.837810 18286 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:14.753252 18295 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:14.763321 18343 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:14.753635 18382 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.860054 18022 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.863428 18200 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.864506 18200 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.756880 18300 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:14.867148 18300 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:14.868176 18437 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:18:14.869185 18187 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.870819 16667 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.764513 18380 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:14.872442 18380 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.873692 16589 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:14.741369 18297 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:14.855223 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:14.884169 17415 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:14.855944 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.856523 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.816394 18443 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:14.860616 17413 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:14.861171 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:14.878389 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:14.847908 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.848858 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:14.882061 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:14.907761 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.851423 18295 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:14.923959 18238 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:14.937882 16666 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.938701 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.944207 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:14.951839 16590 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:14.952690 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.934602 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:14.892724 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:14.992632 16744 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.958292 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:14.962725 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:14.967361 16664 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:14.866814 18022 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:14.970749 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 121970us
W20260709 13:18:14.970773 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.973402 16742 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:18:15.035943 16742 pending_rounds.cc:85] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 2
I20260709 13:18:15.036226 16742 pending_rounds.cc:107] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:18:15.036567 16742 raft_consensus.cc:2889] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.037465 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 777) took 1058 ms. Trace:
I20260709 13:18:14.977087 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2063) took 1022 ms. Trace:
I20260709 13:18:14.872717 18386 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:14.845425 18310 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:18:14.994601 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:15.043715 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49194.1us
I20260709 13:18:14.979261 18295 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:14.998582 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:14.882354 18235 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:18:15.039034 16592 rpcz_store.cc:276] 0709 13:18:13.954474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.022494 (+ 68020us) service_pool.cc:224] Handling call
0709 13:18:14.969662 (+947168us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54002378d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:14.977029 (+  7367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966560512}
I20260709 13:18:15.037992 16742 rpcz_store.cc:276] 0709 13:18:13.978969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.051400 (+ 72431us) service_pool.cc:224] Handling call
0709 13:18:14.051634 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.973078 (+921444us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b54002da8a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb5bdc0 00007f327fbe6d87 00007f327fbe7470 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c
0709 13:18:15.037144 (+ 64066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.037413 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174724352}
I20260709 13:18:14.826856 18385 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:15.041138 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 497500us
I20260709 13:18:14.893638 18443 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:15.052999 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:15.074501 16739 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:14.983000 18383 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:15.060809 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:15.076551 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:15.063024 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:15.007601 18309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.050s	sys 0.108s
I20260709 13:18:14.977567 16740 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:15.043913 18306 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.085961 17415 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:15.086714 18270 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:15.087442 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:15.087792 18424 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:15.088882 16590 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:15.089865 16664 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:15.090080 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:15.090706 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:15.093218 16592 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:15.100838 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:15.069255 18421 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:18:15.059084 17996 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:15.080453 18309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.050s	sys 0.109s
I20260709 13:18:15.043762 16663 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:15.063680 18205 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:15.070261 18187 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:15.133121 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 382616us
I20260709 13:18:15.139941 16592 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:15.082123 18192 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:15.141707 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51069.6us
I20260709 13:18:15.080616 18383 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:15.068608 18022 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:18:15.157682 16666 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:15.146852 16738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.008s	sys 0.004s
I20260709 13:18:15.105646 18444 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:18:15.155737 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:15.124480 18378 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.078055 18177 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.164469 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:15.151381 18308 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.138016 16740 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:15.167909 18310 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:18:15.113238 18306 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:15.170814 18438 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:18:15.172008 18444 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:15.172751 18336 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.147612 18193 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:15.157991 18323 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.002s
W20260709 13:18:15.162456 16738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.009s	sys 0.004s
I20260709 13:18:15.151314 17996 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:15.157096 18383 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.233625 16665 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:15.157135 18346 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 13:18:15.157625 18205 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.252550 17419 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 13:18:15.253405 17419 pending_rounds.cc:85] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 2
I20260709 13:18:15.253783 17419 pending_rounds.cc:107] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 13:18:15.254138 17419 raft_consensus.cc:2889] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.167903 18351 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:15.237311 18444 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:15.186905 18308 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.239029 18207 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.265626 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.255832 17412 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:15.194496 18443 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.199966 18425 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:18:15.244915 18193 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:15.239369 18323 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:15.308970 18240 consensus_meta.cc:287] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.862s	user 0.012s	sys 0.000s
I20260709 13:18:15.301811 18207 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.308938 18386 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.313066 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:15.317377 18443 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.310400 17996 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.320109 18351 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.338239 18097 consensus_meta.cc:287] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.000s
I20260709 13:18:15.340438 18308 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.343804 18205 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:15.371995 18300 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.001s
I20260709 13:18:15.377398 16590 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:18:15.378892 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.328707 18240 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:15.392023 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 288632us
I20260709 13:18:15.323104 18147 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:15.402268 17376 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:15.393100 18450 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.387616 18306 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.409497 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2078) took 1223 ms. Trace:
I20260709 13:18:15.411252 16593 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:15.412122 16596 rpcz_store.cc:276] 0709 13:18:14.185687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.941475 (+755788us) service_pool.cc:224] Handling call
0709 13:18:14.941772 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.942016 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:15.268518 (+326502us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.408437 (+139919us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.408507 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:15.408565 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.409265 (+   700us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.409426 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1651,"thread_start_us":443039,"threads_started":2}
I20260709 13:18:15.338909 18193 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.419715 16664 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:15.407833 18324 consensus_meta.cc:287] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.004s
I20260709 13:18:15.464462 18147 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.464402 18324 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.377187 18351 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.465813 16595 raft_consensus.cc:2368] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5.
I20260709 13:18:15.410840 18421 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.473168 16590 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
W20260709 13:18:15.470542 18380 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.598s	user 0.003s	sys 0.008s
W20260709 13:18:15.454659 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 375588us
I20260709 13:18:15.419390 18450 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.456583 18207 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.320189 18444 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.345834 18343 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.459198 18457 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.467149 18351 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.378340 18097 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.401722 18300 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.468750 18448 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:18:15.500701 18340 consensus_meta.cc:287] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.952s	user 0.010s	sys 0.003s
I20260709 13:18:15.572312 18444 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:15.498495 18421 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:15.466836 18324 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.576501 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:15.577062 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:15.577157 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:15.577953 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:15.578383 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:15.580711 18097 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.586685 18300 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:15.587821 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:15.600440 18457 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.588460 17414 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:18:15.589378 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:15.590432 16745 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:15.593369 16591 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:15.601397 17420 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.573369 18448 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:15.600687 18343 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:15.604096 17413 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:15.633773 18457 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.635568 18450 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.598963 18207 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:18:15.642973 17414 pending_rounds.cc:85] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 5
I20260709 13:18:15.658782 17414 pending_rounds.cc:107] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:18:15.659245 17414 raft_consensus.cc:2889] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:15.635731 18240 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.597149 18421 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:15.652943 18297 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.762s	user 0.010s	sys 0.003s
I20260709 13:18:15.594604 18340 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:15.674832 16636 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.011s	sys 0.037s
W20260709 13:18:15.675266 16636 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.012s	sys 0.037s
W20260709 13:18:15.675557 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 506355us
I20260709 13:18:15.679405 17420 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
W20260709 13:18:15.681067 16636 connection.cc:256] RPC call timeout handler was delayed by -1.1804051730000538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
W20260709 13:18:15.681653 16636 connection.cc:256] RPC call timeout handler was delayed by -1.1576493799998389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 13:18:15.683594 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:15.672204 18297 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:15.678617 16740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.010s	sys 0.019s
I20260709 13:18:15.687541 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:15.690685 16671 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:15.693199 16671 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:15.695223 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.031s	sys 0.048s
I20260709 13:18:15.695827 16671 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:15.682209 18023 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.698685 16671 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:18:15.695616 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.031s	sys 0.048s
I20260709 13:18:15.701087 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:18:15.688647 16740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.011s	sys 0.019s
I20260709 13:18:15.686653 18207 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.708493 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:15.709870 16665 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:15.712651 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:15.713560 16662 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:15.715962 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:15.717794 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:15.701619 18450 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:15.707619 18295 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.629s	user 0.006s	sys 0.006s
I20260709 13:18:15.706756 18300 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:15.745003 16709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.028s	sys 0.058s
W20260709 13:18:15.747006 16709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.028s	sys 0.058s
I20260709 13:18:15.702338 18457 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:15.702788 18297 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:15.734767 16737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.009s	sys 0.015s
I20260709 13:18:15.729522 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:15.748288 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:15.753330 16737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.009s	sys 0.015s
I20260709 13:18:15.755424 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:15.756654 16594 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.000s	sys 0.028s
W20260709 13:18:15.721606 18192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.017s	sys 0.036s
I20260709 13:18:15.671662 18448 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:15.752418 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.759506 16594 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.000s	sys 0.029s
I20260709 13:18:15.705065 18023 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:15.765849 18192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.017s	sys 0.037s
I20260709 13:18:15.775504 18192 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:15.765405 18431 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:15.771258 18309 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:18:15.772753 18382 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.913s	user 0.004s	sys 0.008s
I20260709 13:18:15.765455 18022 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:15.766096 18324 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:15.774508 18448 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:15.764391 18425 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:15.721982 18207 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.736162 18450 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.743085 18295 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.779148 18466 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:15.777143 18240 consensus_queue.cc:1175] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 1 status { last_received { term: 1 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 1 index: 4 } }
I20260709 13:18:15.765554 18434 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:18:15.756808 18300 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.804957 18468 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:15.768620 18457 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:15.808931 18425 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:15.776816 18297 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.815766 18022 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:15.800132 18382 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:15.848927 18439 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.175s	user 0.006s	sys 0.027s
W20260709 13:18:15.859018 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 265974us
I20260709 13:18:15.816089 18324 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.891885 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:15.892537 16666 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:15.850723 18425 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:15.897085 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:15.840222 18473 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:15.902493 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:15.821647 18421 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:15.888967 18439 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.006s	sys 0.028s
I20260709 13:18:15.909363 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2089) took 1146 ms. Trace:
I20260709 13:18:15.907235 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:15.877604 18023 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:15.912333 16596 rpcz_store.cc:276] 0709 13:18:14.762459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.420851 (+658392us) service_pool.cc:224] Handling call
0709 13:18:15.421212 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.421495 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:15.835896 (+414401us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.905367 (+ 69471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:15.905666 (+   299us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:15.905944 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.906165 (+   221us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:15.909079 (+  2914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52199,"prepare.queue_time_us":182327,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":394,"thread_start_us":405538,"threads_started":2}
I20260709 13:18:15.860513 18479 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:15.889873 18340 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.924885 18300 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.841535 18466 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.941180 18295 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:15.914793 18250 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:15.917701 18198 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:15.916553 18457 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.916103 18382 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.928072 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:15.929847 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 174531us
I20260709 13:18:15.944401 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:15.942683 18198 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.947721 18198 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.931262 17419 raft_consensus.cc:2368] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:15.914810 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:16.001940 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57691.6us
I20260709 13:18:16.003015 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:16.004575 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:15.927095 18022 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.932799 18297 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:15.933280 18439 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:15.933607 18450 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:15.903259 17996 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:15.847388 18468 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:15.940327 18444 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:15.943482 18466 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:15.976128 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:15.983250 18443 consensus_meta.cc:287] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.006s
I20260709 13:18:15.978616 18351 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:15.985013 18383 consensus_meta.cc:287] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.001s
I20260709 13:18:15.918465 18309 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.005961 17414 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:18:16.013092 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.019642 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2077) took 1834 ms. Trace:
W20260709 13:18:16.018253 16590 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.014s	sys 0.019s
I20260709 13:18:16.036866 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 148) took 1503 ms. Trace:
I20260709 13:18:16.191000 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2076) took 2006 ms. Trace:
I20260709 13:18:16.189721 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 155) took 1117 ms. Trace:
I20260709 13:18:16.015281 18382 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.158326 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:16.157419 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.197113 16587 rpcz_store.cc:276] 0709 13:18:14.185002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.791248 (+606246us) service_pool.cc:224] Handling call
0709 13:18:14.791847 (+   599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.792311 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.792365 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.793018 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.179731 (+1386713us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b5400260d20. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fb5b6b8 00007f327fbe492b 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c
0709 13:18:16.179843 (+   112us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:16.179922 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.180037 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.180102 (+    65us) raft_consensus.cc:1685] finished
0709 13:18:16.180123 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.190918 (+ 10795us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26464,"raft.queue_time_us":4893,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":14738,"spinlock_wait_cycles":162333056,"thread_start_us":211834,"threads_started":1,"wal-append.queue_time_us":208280,"wal-append.run_cpu_time_us":2214,"wal-append.run_wall_time_us":1099815}
I20260709 13:18:16.201592 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:16.197849 16594 rpcz_store.cc:276] 0709 13:18:14.185374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.815196 (+629822us) service_pool.cc:224] Handling call
0709 13:18:14.815621 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:14.815871 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:15.771502 (+955631us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.019322 (+247820us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.019379 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:16.019417 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.019482 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.019589 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":657271,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":153,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":54671,"thread_start_us":1167375,"threads_started":2}
I20260709 13:18:16.014823 18300 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.154680 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.070212 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 912) took 1104 ms. Trace:
W20260709 13:18:16.152652 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.196252 16745 rpcz_store.cc:276] 0709 13:18:14.533174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.995917 (+462743us) service_pool.cc:224] Handling call
0709 13:18:14.996286 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.660562 (+664276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.036486 (+375924us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400427554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.036777 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1365783296}
W20260709 13:18:16.231889 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73667.6us
W20260709 13:18:16.232789 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.231600 16666 rpcz_store.cc:276] 0709 13:18:14.966085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.159351 (+193266us) service_pool.cc:224] Handling call
0709 13:18:15.171399 (+ 12048us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.898581 (+727182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.069764 (+171183us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b540023d054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.070079 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1657939328}
I20260709 13:18:16.197901 16736 rpcz_store.cc:276] 0709 13:18:15.072773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.472715 (+399942us) service_pool.cc:224] Handling call
0709 13:18:15.473000 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.473171 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:15.881152 (+407981us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.188860 (+307708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:16.188944 (+    84us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.189008 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.189462 (+   454us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.189641 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":676399,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":65,"raft.queue_time_us":95,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":6329,"thread_start_us":714729,"threads_started":2}
I20260709 13:18:16.235597 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.156927 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:16.232887 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:16.203027 16590 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.022s	sys 0.032s
W20260709 13:18:16.153833 18475 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.236s	user 0.002s	sys 0.000s
I20260709 13:18:16.025826 18297 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:16.178329 18471 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.001s
I20260709 13:18:16.051407 18324 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:16.267457 18324 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:16.267586 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.268039 18324 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:16.038841 18478 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:18:16.031265 18469 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
W20260709 13:18:16.270458 18465 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.455s	user 0.001s	sys 0.001s
I20260709 13:18:16.104089 18253 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:16.271073 18253 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:16.272069 18253 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.273166 18253 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.198915 18351 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:16.224607 18383 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.104919 18418 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.275568 18351 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.276429 18418 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.276803 18351 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.277107 17415 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:16.241761 18238 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.278682 18323 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:16.279201 18464 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.505s	user 0.002s	sys 0.000s
I20260709 13:18:16.279310 18238 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.032936 18207 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.258471 18434 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:16.182157 18431 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:16.190546 18474 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:16.259364 18494 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:18:16.105219 18466 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.259562 18340 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.260008 18300 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.260281 18468 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:16.261154 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 104383us
I20260709 13:18:16.264011 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:16.053768 18421 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.028271 18477 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:16.032277 18454 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:16.265949 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 152) took 1258 ms. Trace:
I20260709 13:18:16.159616 18411 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:16.323863 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 429416us
I20260709 13:18:16.199308 18240 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.258083 18488 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:18:16.243345 16736 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:16.278509 18418 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.286494 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.159425 18491 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.288173 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.203928 18457 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.239319 18479 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.281179 18309 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:18:16.292191 18235 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.338373 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:16.077114 18425 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.341013 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:16.079267 18450 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.263s
I20260709 13:18:16.342721 18425 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:16.254590 18297 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.343617 18297 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:16.284231 16708 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.846s (SENT)
I20260709 13:18:16.276671 18383 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.293957 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 154) took 1244 ms. Trace:
I20260709 13:18:16.294657 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 151) took 1287 ms. Trace:
I20260709 13:18:16.296017 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 803) took 1419 ms. Trace:
I20260709 13:18:16.242367 18193 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:16.242022 18363 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.187743 18097 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 13:18:16.241675 18381 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.348903 16737 rpcz_store.cc:276] 0709 13:18:14.876282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.877681 (+  1399us) service_pool.cc:224] Handling call
0709 13:18:14.877954 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.883528 (+  5574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.883555 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.892680 (+  9125us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.753835 (+861155us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:15.753874 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.753944 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.295800 (+541856us) raft_consensus.cc:1685] finished
0709 13:18:16.295822 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.295960 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4329,"raft.queue_time_us":10579,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":8248,"thread_start_us":771720,"threads_started":1,"wal-append.queue_time_us":1034706}
I20260709 13:18:16.347745 16741 rpcz_store.cc:276] 0709 13:18:15.006969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.089342 (+ 82373us) service_pool.cc:224] Handling call
0709 13:18:15.089648 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:15.097330 (+  7682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.097370 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:15.126550 (+ 29180us) log.cc:844] Serialized 52 byte log entry
0709 13:18:15.417498 (+290948us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:15.418531 (+  1033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.418644 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.294295 (+875651us) raft_consensus.cc:1685] finished
0709 13:18:16.294314 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.294601 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7238,"raft.queue_time_us":7865,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":32480,"thread_start_us":258754,"threads_started":1,"wal-append.queue_time_us":634679}
I20260709 13:18:16.242194 18147 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.350941 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.351166 18147 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.351521 16737 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.351331 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.352154 16741 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:16.353181 18147 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.238886 18198 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.353925 18198 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.356062 18198 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.309814 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.311820 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:16.357086 16670 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:16.357659 17414 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:16.358186 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.359252 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:16.318185 16666 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:16.270301 18324 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:16.284577 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:16.305917 18431 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:16.268370 16744 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:16.320513 18300 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.328450 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:16.330670 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2028) took 2617 ms. Trace:
I20260709 13:18:16.336091 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.344400 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:16.345881 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.341070 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.347628 16743 rpcz_store.cc:276] 0709 13:18:15.049902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.267154 (+217252us) service_pool.cc:224] Handling call
0709 13:18:15.304950 (+ 37796us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.722404 (+417454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.722653 (+   249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.738550 (+ 15897us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.102179 (+363629us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:16.102292 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.102445 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.293674 (+191229us) raft_consensus.cc:1685] finished
0709 13:18:16.293702 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.293884 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3764,"raft.queue_time_us":259896,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":6793,"thread_start_us":774493,"threads_started":2,"wal-append.queue_time_us":553290}
I20260709 13:18:16.325347 18477 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:16.350302 18381 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.309077 18466 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.361007 17411 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 1 LEARNER]: Advancing to term 3
I20260709 13:18:16.365971 18454 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.365795 16737 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
W20260709 13:18:16.368827 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 84632.8us
I20260709 13:18:16.368207 16736 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:16.348711 18421 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.363353 18363 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.388834 18363 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:16.332752 18411 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.332227 18240 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.367945 16738 rpcz_store.cc:276] 0709 13:18:13.704747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:13.853210 (+148463us) service_pool.cc:224] Handling call
0709 13:18:13.853688 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.446878 (+593190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.457670 (+ 10792us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.463521 (+  5851us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.239144 (+775623us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:15.243665 (+  4521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.290986 (+1047321us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b5400306954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.305253 (+ 14267us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.305324 (+    71us) raft_consensus.cc:1685] finished
0709 13:18:16.305346 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.321757 (+ 16411us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23177,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":1041113856,"thread_start_us":842850,"threads_started":2,"wal-append.queue_time_us":840212,"wal-append.run_cpu_time_us":2070,"wal-append.run_wall_time_us":1035087}
I20260709 13:18:16.393292 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.393967 16742 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.363482 18193 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:16.342733 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:16.403999 16741 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:16.373858 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.396325 18418 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:16.381844 18383 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.374003 18477 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.401826 18443 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:16.320912 18340 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:16.348212 18235 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.347415 18457 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:16.426090 18427 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:16.426663 18427 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:16.427462 18235 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.427853 18457 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.344576 18491 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.427769 18427 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.429735 18427 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.424532 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 397) took 1325 ms. Trace:
I20260709 13:18:16.292418 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1611) took 1910 ms. Trace:
I20260709 13:18:16.293005 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2079) took 2083 ms. Trace:
I20260709 13:18:16.296466 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2081) took 1799 ms. Trace:
I20260709 13:18:16.847954 18300 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.848340 16592 rpcz_store.cc:276] 0709 13:18:14.497373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.153188 (+655815us) service_pool.cc:224] Handling call
0709 13:18:15.153560 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.165917 (+ 12357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.165946 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.195723 (+ 29777us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.675890 (+480167us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:15.675947 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.678220 (+  2273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.077192 (+398972us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:16.285283 (+208091us) raft_consensus.cc:1685] finished
0709 13:18:16.285312 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.296405 (+ 11093us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14328,"raft.queue_time_us":13773,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":53290,"spinlock_wait_cycles":18609536,"thread_start_us":428723,"threads_started":1,"wal-append.queue_time_us":580163}
W20260709 13:18:16.361304 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49598.7us
I20260709 13:18:16.394548 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.404892 16744 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:16.852473 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2038) took 1152 ms. Trace:
I20260709 13:18:16.852957 16744 rpcz_store.cc:276] 0709 13:18:15.700061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.373607 (+673546us) service_pool.cc:224] Handling call
0709 13:18:16.852239 (+478632us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:16.377887 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49613.7us
I20260709 13:18:16.855371 16742 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:16.473037 18340 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.465590 18380 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:16.857328 18380 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:16.859089 18380 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.859391 18424 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:16.416427 18418 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.859450 18383 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:18:16.861220 18380 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:16.861369 18424 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:16.474417 18381 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.861907 18424 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:16.862076 18383 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:16.862416 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.862583 18383 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:16.862965 18418 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.475937 18411 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.864490 16737 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.865136 18383 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.866107 18240 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.395570 18421 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.459082 18466 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.468161 18474 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:16.867894 18383 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:16.868258 18474 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.419278 18443 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:16.869670 18351 consensus_meta.cc:287] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.004s
W20260709 13:18:16.466974 18476 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.692s	user 0.000s	sys 0.003s
I20260709 13:18:16.870478 18443 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.872576 18443 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:16.471683 18460 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.661s	user 0.003s	sys 0.000s
I20260709 13:18:16.485188 18477 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:16.469987 18496 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 13:18:16.891177 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 900) took 1958 ms. Trace:
I20260709 13:18:16.780655 18450 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:16.569196 16595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1607) took 2187 ms (client timeout 1846 ms). Trace:
I20260709 13:18:16.578311 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 911) took 1612 ms. Trace:
I20260709 13:18:16.589107 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:16.565404 16591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1618) took 2180 ms (client timeout 2212 ms). Trace:
I20260709 13:18:16.568408 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2074) took 2437 ms. Trace:
W20260709 13:18:16.686424 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:16.643605 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 261144us
I20260709 13:18:16.751598 18235 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:16.588506 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.530066 16493 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:16.753141 18177 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.658795 16736 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:16.683899 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.606494 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.798549 17415 rpcz_store.cc:276] 0709 13:18:15.099368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.101327 (+  1959us) service_pool.cc:224] Handling call
0709 13:18:15.101648 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:16.277565 (+1175917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.424163 (+146598us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005492d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.424445 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2702853888}
I20260709 13:18:16.485733 18431 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:16.627130 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.742448 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.741508 17376 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:16.669555 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.707891 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.840144 18479 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:16.840754 18363 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.841164 18425 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:16.841678 18336 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:16.842239 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.842495 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.842990 18457 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.843827 18238 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.324218 16740 rpcz_store.cc:276] 0709 13:18:15.007227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.175446 (+168219us) service_pool.cc:224] Handling call
0709 13:18:15.175831 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.176234 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.176284 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.176847 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.708534 (+531687us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:15.708597 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.722209 (+ 13612us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.265644 (+543435us) raft_consensus.cc:1685] finished
0709 13:18:16.265674 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.265864 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1740,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":45381,"thread_start_us":526705,"threads_started":1,"wal-append.queue_time_us":773732}
I20260709 13:18:16.369146 16667 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:16.859411 18198 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:16.864389 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:16.867522 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.931041 16745 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:16.932037 18240 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.935134 16662 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:16.935532 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2047) took 1209 ms. Trace:
I20260709 13:18:16.435479 18491 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:16.936347 16669 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:16.937199 18491 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:16.934432 18240 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.937491 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.937865 18491 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:16.464238 18493 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:18:16.938941 16589 rpcz_store.cc:276] 0709 13:18:14.209542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.874585 (+665043us) service_pool.cc:224] Handling call
0709 13:18:14.901305 (+ 26720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.901526 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.901552 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.982687 (+ 81135us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.401535 (+418848us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:15.401585 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.401657 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.278582 (+876925us) spinlock_profiling.cc:243] Waited 65.5 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:16.280285 (+  1703us) raft_consensus.cc:1685] finished
0709 13:18:16.280310 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.292931 (+ 12621us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150744960,"thread_start_us":330247,"threads_started":1,"wal-append.queue_time_us":734620}
W20260709 13:18:16.885201 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.885589 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:16.847321 16588 rpcz_store.cc:276] 0709 13:18:14.382503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.979531 (+597028us) service_pool.cc:224] Handling call
0709 13:18:14.979922 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.980340 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.980389 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.980934 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.399931 (+418997us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:15.756315 (+356384us) log.cc:844] Serialized 14 byte log entry
0709 13:18:15.758283 (+  1968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.758382 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.269743 (+511361us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:16.281669 (+ 11926us) raft_consensus.cc:1685] finished
0709 13:18:16.292200 (+ 10531us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.292357 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4576,"prepare.queue_time_us":203661,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":2630,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":27325,"spinlock_wait_cycles":54749440,"thread_start_us":677729,"threads_started":2,"wal-append.queue_time_us":735624}
I20260709 13:18:16.887423 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.886636 18239 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:16.942239 18239 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:16.942700 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.943341 16665 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:16.888615 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.944569 16741 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:16.944885 18239 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.889684 18343 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:16.892148 16595 rpcz_store.cc:269] 0709 13:18:14.381279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.476309 (+ 95030us) service_pool.cc:224] Handling call
0709 13:18:16.568728 (+2092419us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b5400239bd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:16.569004 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2269615488}
I20260709 13:18:16.903798 16737 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:16.945708 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.945761 18343 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.947140 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 142) took 1243 ms. Trace:
I20260709 13:18:16.947427 18450 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:16.947188 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 206111us
I20260709 13:18:16.892438 16664 rpcz_store.cc:276] 0709 13:18:14.965754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.152954 (+187200us) service_pool.cc:224] Handling call
0709 13:18:15.153342 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.457676 (+304334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.577470 (+1119794us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540001b154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.577953 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612132096}
I20260709 13:18:16.949774 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:16.892823 16591 rpcz_store.cc:269] 0709 13:18:14.384489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.684137 (+299648us) service_pool.cc:224] Handling call
0709 13:18:16.564638 (+1880501us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54002373d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:16.565052 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998416128}
W20260709 13:18:16.906692 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.915571 16738 raft_consensus.cc:2368] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
W20260709 13:18:16.916950 16560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.039s	sys 0.051s
W20260709 13:18:16.917372 16590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.030s	sys 0.065s
I20260709 13:18:16.921418 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:16.922169 18235 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:16.917227 16493 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:16.928097 18198 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:16.941205 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:16.948827 18468 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:16.893090 16593 rpcz_store.cc:276] 0709 13:18:14.130437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.273470 (+143033us) service_pool.cc:224] Handling call
0709 13:18:14.608098 (+334628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.412366 (+804268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.565981 (+1153615us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.567847 (+  1866us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b540010d450. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:16.568112 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2615522560}
I20260709 13:18:16.948235 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2063) took 1031 ms. Trace:
I20260709 13:18:16.935958 16736 rpcz_store.cc:276] 0709 13:18:15.708065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.335819 (+627754us) service_pool.cc:224] Handling call
0709 13:18:16.912788 (+576969us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:16.992054 18468 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:16.993468 18468 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.994784 16667 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
W20260709 13:18:16.985479 16560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.046s	sys 0.060s
I20260709 13:18:16.985941 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2037) took 1286 ms. Trace:
I20260709 13:18:16.990729 16737 rpcz_store.cc:276] 0709 13:18:15.916968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.394296 (+477328us) service_pool.cc:224] Handling call
0709 13:18:16.948022 (+553726us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b54000c49d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:16.948175 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29067136}
W20260709 13:18:16.919633 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.017s	sys 0.023s
I20260709 13:18:16.935045 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.986714 18198 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:18:16.994961 16590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.030s	sys 0.066s
I20260709 13:18:17.011247 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.011973 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2094) took 2029 ms. Trace:
I20260709 13:18:17.012753 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.012938 18457 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.014204 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.015166 16739 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.015806 18457 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:16.987555 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:16.998032 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.000510 16662 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:17.001052 18477 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:17.044673 16670 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:17.001242 16493 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 3, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.002393 18193 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.003438 18336 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.068676 18336 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:16.996523 18363 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.071101 18363 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:17.005704 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:17.012437 16590 rpcz_store.cc:276] 0709 13:18:14.982734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.502815 (+520081us) service_pool.cc:224] Handling call
0709 13:18:15.503145 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:15.503354 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:16.258573 (+755219us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.011371 (+752798us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.011467 (+    96us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.011524 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.011650 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.011878 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100680,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":8778,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"thread_start_us":1491325,"threads_started":2}
I20260709 13:18:17.008818 18198 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:17.018533 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:16.891752 16668 rpcz_store.cc:276] 0709 13:18:14.932356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:14.953305 (+ 20949us) service_pool.cc:224] Handling call
0709 13:18:14.953707 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:14.954169 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:14.954210 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:14.954782 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 13:18:15.379863 (+425081us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:15.379906 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:15.379980 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.470691 (+1090711us) spinlock_profiling.cc:243] Waited 35.4 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:16.890512 (+419821us) raft_consensus.cc:1685] finished
0709 13:18:16.890561 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.890943 (+   382us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2002,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":94,"spinlock_wait_cycles":81514368,"thread_start_us":424638,"threads_started":1,"wal-append.queue_time_us":768367}
W20260709 13:18:17.019553 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 99449.8us
I20260709 13:18:17.020684 18235 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.020537 18365 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:17.021385 18421 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.021672 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:17.024927 17411 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.009s
I20260709 13:18:17.025424 18479 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:17.088231 18479 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.089483 16742 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:17.030429 18238 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:17.091333 18479 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.998814 18425 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.091584 16493 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.092398 18365 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:17.093015 18365 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:17.093571 18425 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.979904 16664 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:17.096603 16664 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.097590 16741 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:17.098124 16559 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:17.099179 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 810) took 1027 ms. Trace:
I20260709 13:18:17.099591 16741 rpcz_store.cc:276] 0709 13:18:16.071644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.945303 (+873659us) service_pool.cc:224] Handling call
0709 13:18:17.099004 (+153701us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001c7354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:17.099120 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347397888}
I20260709 13:18:17.047060 18424 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:16.949905 18450 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:16.996098 16738 rpcz_store.cc:276] 0709 13:18:15.699750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.668874 (+969124us) service_pool.cc:224] Handling call
0709 13:18:16.985891 (+317017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.101052 18424 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.027644 18431 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:17.072482 16740 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:17.101931 18431 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:17.102703 18431 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.103619 18198 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.104832 18431 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:17.035959 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.059417 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:17.106102 17420 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:17.060786 18351 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:17.086338 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.091226 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:17.035503 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.018s	sys 0.023s
W20260709 13:18:17.109985 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.106833 16739 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:17.115381 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 806) took 1188 ms. Trace:
I20260709 13:18:17.094295 16493 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.119081 16742 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:17.120035 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.120404 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:17.121201 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 815) took 1048 ms. Trace:
I20260709 13:18:17.121537 16742 rpcz_store.cc:276] 0709 13:18:16.073250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.120074 (+1046824us) service_pool.cc:224] Handling call
0709 13:18:17.121126 (+  1052us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.122556 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:17.124398 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:17.124915 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:17.122576 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:17.119081 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.030329 18343 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:17.132077 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:16.947599 16671 rpcz_store.cc:276] 0709 13:18:15.703165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.703289 (+   124us) service_pool.cc:224] Handling call
0709 13:18:15.703677 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:15.777319 (+ 73642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:15.777347 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:15.777825 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.133825 (+356000us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:16.133969 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.463431 (+329462us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:16.617044 (+153613us) spinlock_profiling.cc:243] Waited 73.2 ms on lock 0x7b54000f2dd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:16.617158 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:16.940447 (+323289us) raft_consensus.cc:1685] finished
0709 13:18:16.940468 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:16.946952 (+  6484us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":730,"raft.run_cpu_time_us":4654,"raft.run_wall_time_us":36104,"spinlock_wait_cycles":174571648,"thread_start_us":311891,"threads_started":1,"wal-append.queue_time_us":500670}
I20260709 13:18:17.115852 16739 rpcz_store.cc:276] 0709 13:18:15.926516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.921014 (+994498us) service_pool.cc:224] Handling call
0709 13:18:17.115293 (+194279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.135353 16741 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:17.136827 16745 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:17.107796 18351 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:17.101928 17411 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:17.141526 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:17.091266 18238 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.084936 18240 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:17.153544 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 808) took 1082 ms. Trace:
I20260709 13:18:17.107934 18235 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:17.158128 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 161) took 1213 ms. Trace:
W20260709 13:18:17.160368 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.145907 16663 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:17.166970 16737 rpcz_store.cc:276] 0709 13:18:15.944973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.031470 (+1086497us) service_pool.cc:224] Handling call
0709 13:18:17.158043 (+126573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.162187 16745 rpcz_store.cc:276] 0709 13:18:16.070968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.741753 (+670785us) service_pool.cc:224] Handling call
0709 13:18:17.153175 (+411422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.159502 18351 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:17.146713 18450 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:17.094395 18365 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.176930 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:17.181762 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:17.170768 16493 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.182893 16670 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.179466 16665 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:17.159937 18336 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:17.186517 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:17.188869 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:17.188072 16736 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.189534 16738 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:17.194594 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2049) took 1450 ms. Trace:
I20260709 13:18:17.195057 16739 rpcz_store.cc:276] 0709 13:18:15.744160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.151290 (+1407130us) service_pool.cc:224] Handling call
0709 13:18:17.194532 (+ 43242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.195847 16744 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:17.198439 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2046) took 1491 ms. Trace:
I20260709 13:18:17.198742 16744 rpcz_store.cc:276] 0709 13:18:15.706976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.141330 (+1434354us) service_pool.cc:224] Handling call
0709 13:18:17.198378 (+ 57048us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:17.205279 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 82877.8us
I20260709 13:18:17.181546 18450 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.172335 18431 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:18:17.211319 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84961.1us
I20260709 13:18:17.206740 18450 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.224942 16742 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
W20260709 13:18:17.227633 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 129630us
W20260709 13:18:17.229718 18427 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.778s	user 0.007s	sys 0.011s
I20260709 13:18:17.230420 18238 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.230530 18501 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:17.207086 18363 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:17.231323 18363 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.233479 18363 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.235246 16590 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:17.236567 18351 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.237289 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.237884 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.238942 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.239303 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.231225 18501 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.208998 18343 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.240899 18343 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.241643 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:17.242439 18501 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.243302 18249 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:17.243188 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.243992 18249 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:17.244724 18427 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:17.245651 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.245160 18249 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.246031 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.245157 18427 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.246731 18427 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.247025 18457 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:17.247437 18457 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.247793 18457 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.248526 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 157) took 2150 ms. Trace:
I20260709 13:18:17.248881 16741 rpcz_store.cc:276] 0709 13:18:15.098326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.191434 (+2093108us) service_pool.cc:224] Handling call
0709 13:18:17.247903 (+ 56469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.248180 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.248207 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.248247 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:17.248296 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.248346 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.248441 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.250602 16741 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:17.251413 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:17.255119 16738 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.243170 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.257104 16745 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:17.240278 16592 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:17.263036 18365 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:17.263041 18431 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.263801 18431 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.265868 18431 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:17.266288 18297 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.917s	user 0.009s	sys 0.004s
I20260709 13:18:17.263237 16636 leader_election.cc:400] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:17.267268 16636 leader_election.cc:403] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:17.230473 18502 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:18:17.267979 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.269387 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 156) took 2172 ms. Trace:
I20260709 13:18:17.269773 16745 rpcz_store.cc:276] 0709 13:18:15.085514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.175648 (+2090134us) service_pool.cc:224] Handling call
0709 13:18:17.217509 (+ 41861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.257538 (+ 40029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.257722 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.271096 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 158) took 1786 ms. Trace:
I20260709 13:18:17.271445 16738 rpcz_store.cc:276] 0709 13:18:15.475463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.200980 (+1725517us) service_pool.cc:224] Handling call
0709 13:18:17.254838 (+ 53858us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.261296 (+  6458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.261564 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.272111 16595 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:17.274511 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2042) took 1561 ms. Trace:
I20260709 13:18:17.274886 16741 rpcz_store.cc:276] 0709 13:18:15.701003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.249959 (+1548956us) service_pool.cc:224] Handling call
0709 13:18:17.250347 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.261719 (+ 11372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.261943 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.259207 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.277105 16662 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:17.205332 18235 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.279245 18454 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:18:17.279687 18253 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.006s	user 0.017s	sys 0.000s
I20260709 13:18:17.279834 16740 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:17.281288 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.277012 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.258759 16493 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:17.284029 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.292567 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.293763 18351 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.294062 16742 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:17.294421 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:17.294778 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2043) took 1593 ms. Trace:
I20260709 13:18:17.295120 17412 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:17.266789 18297 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.296711 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:17.297694 18022 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:17.298053 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:17.298172 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.298137 18022 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.299196 16743 raft_consensus.cc:2368] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3.
I20260709 13:18:17.299293 18022 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.301046 18177 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:17.301457 18177 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.301791 18177 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.303794 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2041) took 1603 ms. Trace:
I20260709 13:18:17.304334 16740 rpcz_store.cc:276] 0709 13:18:15.700783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.273615 (+1572832us) service_pool.cc:224] Handling call
0709 13:18:17.279550 (+  5935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.303446 (+ 23896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.303724 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.306398 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2050) took 1560 ms. Trace:
I20260709 13:18:17.306596 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.307444 16590 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:17.307806 16587 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:17.307978 16591 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:17.308094 18022 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.295197 16742 rpcz_store.cc:276] 0709 13:18:15.701224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.269525 (+1568301us) service_pool.cc:224] Handling call
0709 13:18:17.293831 (+ 24306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.294519 (+   688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.294704 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.310425 16594 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:17.310947 16589 raft_consensus.cc:2410] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:17.311347 16742 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:17.311916 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2051) took 1546 ms. Trace:
I20260709 13:18:17.312166 16745 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:17.312851 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2044) took 1611 ms. Trace:
I20260709 13:18:17.313226 16745 rpcz_store.cc:276] 0709 13:18:15.701456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.270479 (+1569023us) service_pool.cc:224] Handling call
0709 13:18:17.311841 (+ 41362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.312558 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.312804 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:17.313834 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54702.1us
I20260709 13:18:17.312199 16742 rpcz_store.cc:276] 0709 13:18:15.766063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.310668 (+1544605us) service_pool.cc:224] Handling call
0709 13:18:17.311043 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.311693 (+   650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.311862 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.278352 18193 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:17.314925 18453 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:17.318890 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:17.315651 16739 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.321009 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 159) took 1845 ms. Trace:
I20260709 13:18:17.321488 16739 rpcz_store.cc:276] 0709 13:18:15.475792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.206413 (+1730621us) service_pool.cc:224] Handling call
0709 13:18:17.278675 (+ 72262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.320762 (+ 42087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.320953 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.298574 18297 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:17.334828 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.330392 16737 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.336608 16592 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:18:17.337934 16745 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.336074 18192 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:17.273651 18479 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:18:17.339227 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2052) took 1539 ms. Trace:
I20260709 13:18:17.340353 16745 rpcz_store.cc:276] 0709 13:18:15.799462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.313823 (+1514361us) service_pool.cc:224] Handling call
0709 13:18:17.326389 (+ 12566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.338995 (+ 12606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.339182 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.299117 18434 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:17.342644 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2040) took 1638 ms. Trace:
I20260709 13:18:17.338497 16592 pending_rounds.cc:85] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 1
I20260709 13:18:17.343137 16737 rpcz_store.cc:276] 0709 13:18:15.700570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.214194 (+1513624us) service_pool.cc:224] Handling call
0709 13:18:17.285428 (+ 71234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.338484 (+ 53056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.338702 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.312912 18491 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:17.321269 18343 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.321377 18193 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:17.347878 16742 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:17.349838 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2054) took 1537 ms. Trace:
I20260709 13:18:17.350512 16742 rpcz_store.cc:276] 0709 13:18:15.812306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.315783 (+1503477us) service_pool.cc:224] Handling call
0709 13:18:17.333350 (+ 17567us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.349423 (+ 16073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.349757 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.326161 16493 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 2, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 2 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:18:17.324092 18477 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:17.373278 18193 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.373515 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.332720 18431 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:17.375610 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.391049 16591 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.376423 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.378826 18177 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.339926 18192 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:17.379714 16493 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:17.343413 16592 pending_rounds.cc:107] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:18:17.339995 18453 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:17.387634 16666 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
W20260709 13:18:17.388947 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.385923 16588 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:17.393708 16592 raft_consensus.cc:2889] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:17.392571 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 160) took 1577 ms. Trace:
W20260709 13:18:17.383869 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 132615us
I20260709 13:18:17.325702 18474 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.397137 16739 rpcz_store.cc:276] 0709 13:18:15.815162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.341374 (+1526212us) service_pool.cc:224] Handling call
0709 13:18:17.383375 (+ 42001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.383631 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.383665 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.392244 (+  8579us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:17.392291 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.392370 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.392508 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.401312 16737 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:17.402076 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2058) took 1520 ms. Trace:
I20260709 13:18:17.402443 16737 rpcz_store.cc:276] 0709 13:18:15.881549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.348248 (+1466699us) service_pool.cc:224] Handling call
0709 13:18:17.398969 (+ 50721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.401778 (+  2809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.402012 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:17.399863 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 73339us
W20260709 13:18:17.403667 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.407681 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2057) took 1559 ms. Trace:
I20260709 13:18:17.408191 16745 rpcz_store.cc:276] 0709 13:18:15.848039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.347691 (+1499652us) service_pool.cc:224] Handling call
0709 13:18:17.391600 (+ 43909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.406480 (+ 14880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.406521 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.406578 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.406619 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.406694 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.406826 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.338423 18502 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:18:17.390724 18431 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:17.312753 16740 rpcz_store.cc:276] 0709 13:18:15.745979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.305311 (+1559332us) service_pool.cc:224] Handling call
0709 13:18:17.305802 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.306021 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.306060 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.306113 (+    53us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.306151 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.306211 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.306321 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.410566 16589 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:17.405874 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2045) took 1700 ms. Trace:
I20260709 13:18:17.420781 16738 rpcz_store.cc:276] 0709 13:18:15.705735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.278545 (+1572810us) service_pool.cc:224] Handling call
0709 13:18:17.347874 (+ 69329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.405445 (+ 57571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.405486 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.405542 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.405583 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.405659 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.405808 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.399497 16493 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.394830 18192 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:17.426556 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.427278 16740 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.428077 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 807) took 1501 ms. Trace:
I20260709 13:18:17.428759 18457 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.429487 16592 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:17.429065 16493 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:17.431316 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.432148 16587 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:17.432802 18309 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.433373 18309 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.433975 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:17.428455 16740 rpcz_store.cc:276] 0709 13:18:15.926834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.426549 (+1499715us) service_pool.cc:224] Handling call
0709 13:18:17.427062 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.427785 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.428010 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.435964 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:17.436578 17415 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:17.437429 18177 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.437569 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 809) took 1366 ms. Trace:
I20260709 13:18:17.419433 18431 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.438189 16740 rpcz_store.cc:276] 0709 13:18:16.071331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.436234 (+1364903us) service_pool.cc:224] Handling call
0709 13:18:17.436634 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.436832 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.436867 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.437298 (+   431us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.437341 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.437403 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.437516 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.439926 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:17.440672 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:17.441792 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:17.442283 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:17.411034 18477 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:17.460513 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45572.5us
I20260709 13:18:17.463143 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.465384 18502 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.465793 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.465983 18502 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.462248 18177 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:17.467741 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.468353 18502 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:17.456329 18457 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:17.468878 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.469624 16587 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:17.470484 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.435609 18309 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:17.471397 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48160.8us
I20260709 13:18:17.471994 16587 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:17.471643 18477 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.422603 18343 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:17.466432 18434 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:17.485428 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.491382 16589 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:17.457367 18421 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:17.507992 18309 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:17.503754 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 348) took 1006 ms. Trace:
W20260709 13:18:17.513998 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 71831.4us
W20260709 13:18:17.517241 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 132250us
I20260709 13:18:17.475577 18343 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:17.516767 16592 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:17.504393 18502 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.518431 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.527894 16636 leader_election.cc:400] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:17.528283 16594 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:17.521891 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2061) took 1605 ms. Trace:
I20260709 13:18:17.530742 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.531162 16745 rpcz_store.cc:276] 0709 13:18:15.916429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.414515 (+1498086us) service_pool.cc:224] Handling call
0709 13:18:17.459423 (+ 44908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.517116 (+ 57693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.517144 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.521535 (+  4391us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.521589 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.521669 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.521823 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.532624 16745 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:17.533182 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 166) took 1140 ms. Trace:
I20260709 13:18:17.533476 16745 rpcz_store.cc:276] 0709 13:18:16.392462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.532114 (+1139652us) service_pool.cc:224] Handling call
0709 13:18:17.532459 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.532964 (+   505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.533133 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.537708 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.538625 16592 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:17.539718 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.541111 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.542137 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:17.549491 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.549921 16594 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:17.510030 17416 rpcz_store.cc:276] 0709 13:18:16.326662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.362616 (+ 35954us) service_pool.cc:224] Handling call
0709 13:18:16.366394 (+  3778us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:16.371597 (+  5203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:16.371642 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:16.372191 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:18:16.793764 (+421573us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:16.794541 (+   777us) log.cc:844] Serialized 14 byte log entry
0709 13:18:16.850213 (+ 55672us) spinlock_profiling.cc:243] Waited 3.18 ms on lock 0x7b18004019e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:18:16.850512 (+   299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:16.850706 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.315988 (+465282us) raft_consensus.cc:1685] finished
0709 13:18:17.332693 (+ 16705us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.332869 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28725,"prepare.queue_time_us":17651,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":66252,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":315,"spinlock_wait_cycles":7323008,"thread_start_us":418380,"threads_started":1,"wal-append.queue_time_us":848606}
I20260709 13:18:17.503753 18421 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:17.526605 18177 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:17.565714 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.595757 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.558176 18421 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:17.558405 18177 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:17.604792 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 98010.4us
W20260709 13:18:17.604914 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.631s (SENT)
I20260709 13:18:17.613612 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:17.619375 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 89492.5us
I20260709 13:18:17.621474 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.627884 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.562844 18502 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.642107 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:17.649360 18249 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.651261 18383 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:17.690143 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.691027 16594 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:17.653275 18512 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.359s	user 0.003s	sys 0.001s
W20260709 13:18:17.653646 18519 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:17.602411 18198 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:18:17.624466 18479 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:17.656420 16666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1616) took 1367 ms (client timeout 1821 ms). Trace:
I20260709 13:18:17.664388 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.626508 18365 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 13:18:17.688987 18431 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:17.692502 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2059) took 1770 ms. Trace:
I20260709 13:18:17.720631 16742 rpcz_store.cc:276] 0709 13:18:15.891499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.351965 (+1460466us) service_pool.cc:224] Handling call
0709 13:18:17.381271 (+ 29306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.397765 (+ 16494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.397808 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.416912 (+ 19104us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.517211 (+100299us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.517249 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.517329 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.661654 (+144325us) raft_consensus.cc:1685] finished
0709 13:18:17.661681 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.661863 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11565,"raft.queue_time_us":16707,"spinlock_wait_cycles":2526464,"thread_start_us":73906,"threads_started":1,"wal-append.queue_time_us":170610}
I20260709 13:18:17.721663 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.721782 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.659200 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.722540 16662 raft_consensus.cc:2368] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:17.726142 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:17.727066 16662 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:17.728286 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.728964 16662 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:17.729949 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:17.714984 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52455.7us
I20260709 13:18:17.732630 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:17.560298 18453 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:17.713438 16666 rpcz_store.cc:269] 0709 13:18:16.288903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.587708 (+298805us) service_pool.cc:224] Handling call
0709 13:18:17.656111 (+1068403us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540009ea54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:17.656297 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949989248}
W20260709 13:18:17.722481 16743 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2055) took 1901 ms (client timeout 1998 ms). Trace:
I20260709 13:18:17.665649 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.723133 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:17.738796 16594 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:17.639933 18457 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.742002 18457 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:17.737732 16743 rpcz_store.cc:269] 0709 13:18:15.815074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.391180 (+576106us) service_pool.cc:224] Handling call
0709 13:18:17.691176 (+1299996us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54001c7854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:17.715884 (+ 24708us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889097600}
W20260709 13:18:17.719444 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.516s (SENT) [suppressed 10 similar messages]
I20260709 13:18:17.747865 16589 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:18:17.720697 18474 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:17.754470 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 118239us
I20260709 13:18:17.719512 18468 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:18:17.787784 18468 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:17.788178 18518 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:18:17.788830 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 152760us
I20260709 13:18:17.790336 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:17.834311 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:17.651342 18346 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.143s
I20260709 13:18:17.832808 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.753978 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 165) took 1372 ms. Trace:
I20260709 13:18:17.862653 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:17.788208 18468 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:17.838538 18474 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:17.866000 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.837690 18457 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:17.866039 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:17.856966 18501 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:17.900564 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.900754 16595 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
W20260709 13:18:17.865878 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:17.791070 18479 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:17.914948 16742 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:18:17.893926 18512 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:17.923225 18530 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
W20260709 13:18:17.925889 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:17.927621 18521 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 13:18:17.917677 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2067) took 1018 ms. Trace:
I20260709 13:18:17.976228 16742 rpcz_store.cc:276] 0709 13:18:16.905491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.789988 (+884497us) service_pool.cc:224] Handling call
0709 13:18:17.863863 (+ 73875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.917294 (+ 53431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.917599 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:17.947170 18527 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:18:17.981248 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 114416us
W20260709 13:18:17.951169 18522 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:18:17.983103 16593 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:18:17.955338 18523 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:18:17.969625 18235 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:17.982221 16634 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:18.000468 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 167) took 1341 ms. Trace:
I20260709 13:18:17.910843 18457 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:17.921721 18501 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.007283 16742 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:18.000963 16745 rpcz_store.cc:276] 0709 13:18:16.659607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.534217 (+874610us) service_pool.cc:224] Handling call
0709 13:18:17.534520 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.534845 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:17.582167 (+ 47322us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.971783 (+389616us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b18002afdc0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:18.000074 (+ 28291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:18.000136 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:18.000181 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.000249 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.000397 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47289,"prepare.run_wall_time_us":47,"raft.queue_time_us":32250,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51,"spinlock_wait_cycles":13181056,"thread_start_us":358467,"threads_started":2,"wal-append.queue_time_us":342733}
I20260709 13:18:18.010809 16594 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:17.754189 18177 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:17.902849 18474 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:18.011641 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 111746us
W20260709 13:18:18.030634 16736 consensus_meta.cc:287] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.824s	user 0.000s	sys 0.029s
I20260709 13:18:18.042760 16736 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:18.043363 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2069) took 1116 ms. Trace:
I20260709 13:18:18.043681 16738 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:18.043923 16736 rpcz_store.cc:276] 0709 13:18:16.959173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.085342 (+126169us) service_pool.cc:224] Handling call
0709 13:18:18.043302 (+957960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.030453 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2070) took 1035 ms. Trace:
I20260709 13:18:18.043337 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:18.044786 16742 rpcz_store.cc:276] 0709 13:18:16.995500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.977267 (+981767us) service_pool.cc:224] Handling call
0709 13:18:18.006654 (+ 29387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.030015 (+ 23361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.030347 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:18.032904 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 431821us
I20260709 13:18:17.715399 18249 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.049162 16671 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:18.056365 18235 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.057075 18235 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.057464 18235 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.057716 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.056344 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2062) took 2139 ms. Trace:
I20260709 13:18:18.058831 16739 rpcz_store.cc:276] 0709 13:18:15.916736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.411962 (+1495226us) service_pool.cc:224] Handling call
0709 13:18:17.465629 (+ 53667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.465903 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:17.558794 (+ 92891us) log.cc:844] Serialized 14 byte log entry
0709 13:18:18.055863 (+497069us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:18.055926 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:18.055983 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.056060 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.056228 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42154,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":42836,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":98,"spinlock_wait_cycles":117041536,"thread_start_us":528066,"threads_started":3,"wal-append.queue_time_us":260099}
W20260709 13:18:18.063756 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.069199 16666 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:18:18.032646 18501 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.075311 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:18.085081 17416 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:18:18.115741 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.074493 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92413.3us
I20260709 13:18:18.054328 18431 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:18.024945 18457 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.129822 16665 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:18.150803 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2039) took 2450 ms. Trace:
I20260709 13:18:18.154752 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:18.155995 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.126168 18453 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:18.135690 18346 raft_consensus.cc:670] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:18.169049 18538 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.169747 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2048) took 2428 ms. Trace:
I20260709 13:18:18.169669 18538 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.170388 16741 rpcz_store.cc:276] 0709 13:18:15.740594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.275566 (+1534972us) service_pool.cc:224] Handling call
0709 13:18:17.327909 (+ 52343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.390819 (+ 62910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.390847 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.394731 (+  3884us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.547560 (+152829us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b48002307d8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:18:17.571451 (+ 23891us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.571515 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.571618 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.161002 (+589384us) raft_consensus.cc:1685] finished
0709 13:18:18.161028 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.168849 (+  7821us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8053,"raft.run_cpu_time_us":3278,"raft.run_wall_time_us":154384,"spinlock_wait_cycles":43488128,"thread_start_us":144626,"threads_started":1,"wal-append.queue_time_us":208658}
I20260709 13:18:18.172336 18538 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:18.173528 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 383056us
I20260709 13:18:18.175276 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:18.151621 16744 rpcz_store.cc:276] 0709 13:18:15.700319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.208577 (+1508258us) service_pool.cc:224] Handling call
0709 13:18:17.285534 (+ 76957us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.364656 (+ 79122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.364688 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.400356 (+ 35668us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.480538 (+ 80182us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:17.480591 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.480693 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.150582 (+669889us) raft_consensus.cc:1685] finished
0709 13:18:18.150600 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.150739 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41407,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":32957,"thread_start_us":68637,"threads_started":1,"wal-append.queue_time_us":189221}
I20260709 13:18:18.167579 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:18.183044 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.190832 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50101.6us
I20260709 13:18:18.169562 18431 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:18.239094 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:18.245043 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 525536us
I20260709 13:18:17.864494 16740 rpcz_store.cc:276] 0709 13:18:16.377141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.439450 (+1062309us) service_pool.cc:224] Handling call
0709 13:18:17.439861 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.440177 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:17.444801 (+  4624us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.620670 (+175869us) log.cc:844] Serialized 14 byte log entry
0709 13:18:17.660132 (+ 39462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:17.670644 (+ 10512us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b180032e480. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7522 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:17.669378 (+ -1266us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:17.712454 (+ 43076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.715682 (+  3228us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:17.748948 (+ 33266us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":971,"prepare.queue_time_us":65817,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":121,"raft.queue_time_us":41491,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":6084,"spinlock_wait_cycles":100493824,"thread_start_us":113862,"threads_started":1,"wal-append.queue_time_us":193080}
W20260709 13:18:18.250686 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54171.7us
W20260709 13:18:18.244733 18427 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.997s	user 0.013s	sys 0.018s
I20260709 13:18:18.240557 18249 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:18.244989 18502 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.154634 18468 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:18.265295 18351 consensus_meta.cc:287] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.971s	user 0.017s	sys 0.012s
I20260709 13:18:18.291934 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:18.407259 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 139370us
W20260709 13:18:18.275683 18192 consensus_meta.cc:287] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.802s	user 0.011s	sys 0.001s
W20260709 13:18:18.413448 16560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.013s	sys 0.005s
W20260709 13:18:18.424661 16560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.014s	sys 0.005s
I20260709 13:18:18.425122 16665 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:18:18.463369 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55447.7us
I20260709 13:18:18.261436 18431 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.473191 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2071) took 1463 ms. Trace:
W20260709 13:18:18.416457 18380 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.527s	user 0.009s	sys 0.006s
W20260709 13:18:18.478197 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 303035us
I20260709 13:18:18.475239 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.475111 16745 rpcz_store.cc:276] 0709 13:18:17.009913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.031322 (+1021409us) service_pool.cc:224] Handling call
0709 13:18:18.031628 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.135345 (+103717us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:18.176790 (+ 41445us) log.cc:844] Serialized 14 byte log entry
0709 13:18:18.472352 (+295562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:18.472510 (+   158us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:18.472696 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.472840 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.473091 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25368,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4619,"raft.queue_time_us":3766,"thread_start_us":202108,"threads_started":1}
I20260709 13:18:18.276360 18427 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:18.496523 16591 consensus_meta.cc:287] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.096s	user 0.015s	sys 0.015s
I20260709 13:18:18.497301 16591 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:18.498227 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1631) took 1155 ms. Trace:
I20260709 13:18:18.498724 16591 rpcz_store.cc:276] 0709 13:18:17.343120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.375243 (+ 32123us) service_pool.cc:224] Handling call
0709 13:18:18.498147 (+1122904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:18.501081 16634 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:18.484843 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.516628 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.518944 16665 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:18.484156 18502 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:18.485800 18424 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.384s	user 0.010s	sys 0.004s
I20260709 13:18:18.534832 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:18.460181 18192 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.472599 18249 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:18:18.551971 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57516.4us
I20260709 13:18:18.509328 18468 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.544468 17419 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:18.569360 18551 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:18.570544 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.523567 18363 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:18.569082 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.530826 18479 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:18.580646 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.591882 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.552832 18022 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.558043 18424 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.560179 18556 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:18.546010 18240 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:18:18.634073 18546 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:18:18.626482 18380 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.613370 18457 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:18.635869 18537 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.388s	user 0.000s	sys 0.001s
I20260709 13:18:18.624060 18346 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:18.632191 18193 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.232s	user 0.013s	sys 0.001s
I20260709 13:18:18.654770 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:18.658879 16743 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:18.655186 18380 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.655226 18022 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.663631 18022 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.666199 18022 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.655752 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:18.657490 18346 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.668344 17420 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:18.657977 18193 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:18.671494 18193 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.672134 17412 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:18.640199 18192 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:18.635776 18552 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:18:18.674136 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:18.653466 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:18.653767 17415 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:18.675649 16745 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:18.672410 16670 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:18.679085 18380 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:18.680235 18540 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:18.682778 18427 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:18.626044 18556 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.652041 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:18.644933 17416 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.000s
I20260709 13:18:18.688629 17416 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:18.699968 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:18.614327 18363 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.700902 16742 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.705641 18553 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.706952 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.707950 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1628) took 1093 ms. Trace:
I20260709 13:18:18.708362 16663 rpcz_store.cc:276] 0709 13:18:17.614070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.620647 (+  6577us) service_pool.cc:224] Handling call
0709 13:18:18.706758 (+1086111us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:18.707879 (+  1121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12659456}
I20260709 13:18:18.708889 18431 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:18.709826 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.711575 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:18.719295 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 218275us
W20260709 13:18:18.721890 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 102854us
W20260709 13:18:18.718844 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 66953.7us
I20260709 13:18:18.725494 16708 leader_election.cc:400] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:18.629484 18424 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.726840 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:18.727816 16708 leader_election.cc:403] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:18.690356 18556 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:18.701640 18192 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.733964 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:18.727522 18421 consensus_meta.cc:287] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.115s	user 0.014s	sys 0.018s
I20260709 13:18:18.724989 18431 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.722005 18427 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:18.742622 16745 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:18:18.733656 18434 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.218s	user 0.006s	sys 0.008s
I20260709 13:18:18.725293 18380 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:18:18.706331 18553 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.746312 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:18.748641 18443 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:18:18.750784 18560 raft_consensus.cc:2764] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:18:18.745796 18434 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:18.756584 18567 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:18.759456 18543 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.758962 16742 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:18.740120 18431 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.760524 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:18.746483 18346 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:18.751276 18198 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:18.739480 18421 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.764597 18418 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:18.770949 18240 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:18.771862 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 100) took 1135 ms. Trace:
I20260709 13:18:18.742461 18192 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.772346 16592 rpcz_store.cc:276] 0709 13:18:17.636256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.636489 (+   233us) service_pool.cc:224] Handling call
0709 13:18:17.659476 (+ 22987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.710453 (+ 50977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.710481 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.752775 (+ 42294us) log.cc:844] Serialized 26 byte log entry
0709 13:18:18.001994 (+249219us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:18.002034 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.002103 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.771620 (+769517us) raft_consensus.cc:1685] finished
0709 13:18:18.771643 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.771792 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133295,"raft.queue_time_us":49517,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":7394,"thread_start_us":213325,"threads_started":1,"wal-append.queue_time_us":934044}
I20260709 13:18:18.774660 18544 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:18.775514 18533 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.723s	user 0.006s	sys 0.006s
I20260709 13:18:18.776685 18491 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.764834 18543 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:18.760051 18454 consensus_meta.cc:287] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.447s	user 0.016s	sys 0.019s
I20260709 13:18:18.778445 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1626) took 1165 ms. Trace:
I20260709 13:18:18.780651 18346 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:18.782836 16670 rpcz_store.cc:276] 0709 13:18:17.613439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.620922 (+  7483us) service_pool.cc:224] Handling call
0709 13:18:18.778195 (+1157273us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b54001f6b54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:18.778370 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2182924544}
I20260709 13:18:18.785578 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:18.785996 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:18.783427 18543 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.784271 18491 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.769326 18198 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:18.788640 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.790625 17415 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:18.797964 16670 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:18.798456 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:18.791810 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1639) took 1002 ms. Trace:
I20260709 13:18:18.792003 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1634) took 1175 ms. Trace:
I20260709 13:18:18.799355 16671 rpcz_store.cc:276] 0709 13:18:17.616291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.712388 (+ 96097us) service_pool.cc:224] Handling call
0709 13:18:18.791788 (+1079400us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b540001a254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:18.791944 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751122688}
I20260709 13:18:18.792724 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.793572 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:18.793880 18235 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.736s	user 0.005s	sys 0.005s
I20260709 13:18:18.794656 18527 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:18.795943 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 805) took 2875 ms. Trace:
I20260709 13:18:18.770912 18424 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.791116 18491 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:18.801260 16738 rpcz_store.cc:276] 0709 13:18:15.920703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.448983 (+1528280us) service_pool.cc:224] Handling call
0709 13:18:18.043492 (+594509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:18.044130 (+   638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.795674 (+751544us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400059e50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:18.795842 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1260345216}
W20260709 13:18:18.783310 18450 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.510s	user 0.015s	sys 0.019s
I20260709 13:18:18.805778 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:18.813531 16587 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.819078 18450 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.821527 18450 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:18.827148 18238 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:18.827682 18238 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:18.828347 18192 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:18.828830 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:18.828644 18238 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.828367 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:18.829953 18421 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.807006 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1632) took 1191 ms. Trace:
I20260709 13:18:18.808235 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:18.830742 16664 rpcz_store.cc:276] 0709 13:18:17.615678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.626143 (+ 10465us) service_pool.cc:224] Handling call
0709 13:18:18.793429 (+1167286us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:18.806947 (+ 13518us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30400256}
I20260709 13:18:18.810148 16590 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:18.832206 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.832804 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:18.832981 16664 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:18.834125 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:18.810925 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:18.811558 18562 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 13:18:18.835616 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 78624.6us
W20260709 13:18:18.812776 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.836225 18525 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.977s	user 0.009s	sys 0.000s
I20260709 13:18:18.829982 18238 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.831127 16589 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:18.809230 16593 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:18.839376 18553 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.839668 18546 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:18.840093 16593 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:18.845826 18431 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:18.799280 16662 rpcz_store.cc:276] 0709 13:18:17.789273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.837883 (+ 48610us) service_pool.cc:224] Handling call
0709 13:18:17.863232 (+ 25349us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:18.160193 (+296961us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:18.161178 (+   985us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:18.238435 (+ 77257us) log.cc:844] Serialized 52 byte log entry
0709 13:18:18.483986 (+245551us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:18.490895 (+  6909us) log.cc:844] Serialized 14 byte log entry
0709 13:18:18.500895 (+ 10000us) log.cc:844] Serialized 14 byte log entry
0709 13:18:18.624138 (+123243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:18.624239 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:18.638116 (+ 13877us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b1800431aa0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:18.791533 (+153417us) raft_consensus.cc:1685] finished
0709 13:18:18.791559 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:18.791717 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22781,"prepare.queue_time_us":36389,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":184511,"raft.run_cpu_time_us":413,"raft.run_wall_time_us":10918,"spinlock_wait_cycles":52414080,"thread_start_us":536405,"threads_started":3,"wal-append.queue_time_us":433288}
I20260709 13:18:18.784397 18454 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4.
I20260709 13:18:18.840607 18567 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:18.781324 18544 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.840719 18363 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:18.841198 17419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 407) took 1717 ms (client timeout 2141 ms). Trace:
I20260709 13:18:18.840637 16590 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
W20260709 13:18:18.857331 17419 rpcz_store.cc:269] 0709 13:18:17.123922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.124138 (+   216us) service_pool.cc:224] Handling call
0709 13:18:18.840992 (+1716854us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54005d25d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:18.841101 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3264208256}
I20260709 13:18:18.841603 18365 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:18.806593 16669 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:18.834039 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.843413 16668 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:18:18.844225 18235 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.855792 16587 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:18.861469 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:18.862185 18343 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.860213 18567 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:18.863641 16669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1630) took 1249 ms (client timeout 1545 ms). Trace:
W20260709 13:18:18.856495 18501 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.002s
W20260709 13:18:18.872543 16669 rpcz_store.cc:269] 0709 13:18:17.614683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.620750 (+  6067us) service_pool.cc:224] Handling call
0709 13:18:18.792552 (+1171802us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:18.863584 (+ 71032us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39069056}
I20260709 13:18:18.877691 16664 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:18.877499 18365 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:18.886144 18365 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:18.887306 18343 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:18.860354 18249 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.900228 16665 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:18.901561 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:18.873039 18491 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.904307 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:18.888134 18528 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:18.911896 16666 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:18.908320 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:18.877645 18527 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:18.914083 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 67029.6us
I20260709 13:18:18.914274 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.916154 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:18.877290 17412 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:18.912906 16669 raft_consensus.cc:2376] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3. Re-sending same reply.
I20260709 13:18:18.899474 18431 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:18.920106 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:18.920888 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:18.922372 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 58845.7us
I20260709 13:18:18.879280 18501 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.922853 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:18.880515 18567 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.926828 18447 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:18:18.929926 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:18.914637 18491 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.931581 18022 raft_consensus.cc:1064] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:18.918912 18530 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:18.933525 18491 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:18.950106 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:18.951715 18535 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:18.951346 18022 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.953595 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:18.954234 16667 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.956112 18527 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.959980 18477 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:18.961809 18235 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:18.934163 18434 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:18.962888 18434 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.963269 18434 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:18.964427 18177 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:18.965372 18383 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:18.965853 18383 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:18.967566 18535 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:18.962450 18531 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 1.052s	user 0.004s	sys 0.016s
W20260709 13:18:18.968289 18532 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 1.025s	user 0.006s	sys 0.007s
I20260709 13:18:18.968647 18535 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:18.972270 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:18.974794 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49765.1us
I20260709 13:18:18.977275 16591 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:18.978061 16667 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:18.914700 18249 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:18.979768 16667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1631) took 1364 ms (client timeout 1506 ms). Trace:
I20260709 13:18:18.940348 18450 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:18.983269 16667 rpcz_store.cc:269] 0709 13:18:17.615332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.626235 (+ 10903us) service_pool.cc:224] Handling call
0709 13:18:18.953443 (+1327208us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:18.979704 (+ 26261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13032320}
I20260709 13:18:18.919968 18253 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:18:18.946018 18425 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:18.921806 18454 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 13:18:18.997562 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:19.000092 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:18.949697 18530 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.910737 18198 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.916311 18528 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:18.925539 18431 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:18.990890 18297 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:18.994302 18450 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:18.995652 18443 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:19.001703 18425 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:18.998802 18553 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:19.031494 18340 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:19.032459 18542 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:18:19.036114 18249 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.027738 18450 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.008497 18530 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.039145 18431 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:19.039228 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.040434 18501 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.040237 16590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:18:19.041749 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.042330 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.076763 16743 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:19.076865 16745 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:19.043843 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.044236 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.044586 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.044977 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.045229 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.079612 16736 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:19.079800 16741 raft_consensus.cc:2410] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:19.045624 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.018185 18558 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:19.046900 18198 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.046360 18528 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.046327 18383 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.047372 18443 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.102546 16592 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:19.048197 18297 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:19.055342 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.058390 18556 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.104673 16666 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:19.056036 16663 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:19.106117 18383 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.107163 18453 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:19.107631 18453 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:19.108690 18453 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.110118 18453 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.018955 18491 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:19.111089 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.111588 18491 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:19.111835 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.112536 17414 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:19.112649 18491 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.113813 18556 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.063061 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.063771 18340 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.116186 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.116907 17411 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:19.117493 18340 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.118134 18340 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.118429 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:19.119554 17416 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:19.119674 18297 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.056339 18553 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.117514 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.123769 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:19.126353 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.124912 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.063050 16738 raft_consensus.cc:2368] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
I20260709 13:18:19.052146 18425 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.077991 16744 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:19.080379 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:19.082463 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2060) took 3175 ms. Trace:
I20260709 13:18:19.083721 17412 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:19.084353 17419 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:19.081696 16590 raft_consensus.cc:606] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.103899 18558 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:19.085474 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.086695 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.090525 18421 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:19.090987 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:19.094132 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.094193 18542 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:19.073071 18177 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:19.163646 16559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.330s	user 0.014s	sys 0.027s
I20260709 13:18:19.164202 18556 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:19.164011 18177 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.130667 18418 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:19.164800 18556 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.136682 16737 rpcz_store.cc:276] 0709 13:18:15.906991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.403697 (+1496706us) service_pool.cc:224] Handling call
0709 13:18:17.450922 (+ 47225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:17.490967 (+ 40045us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:17.555918 (+ 64951us) log.cc:844] Serialized 14 byte log entry
0709 13:18:19.034763 (+1478845us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b18002ace80. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f78423a 00007f327fbeea2a
0709 13:18:19.082089 (+ 47326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:19.082146 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:19.082200 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:19.082269 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.082401 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":46420,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1373,"raft.queue_time_us":94,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":4992256,"thread_start_us":46483,"threads_started":1}
I20260709 13:18:19.142321 16741 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.143559 16743 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.141530 16590 raft_consensus.cc:614] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:19.130479 18300 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:19.164072 16559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.330s	user 0.014s	sys 0.027s
I20260709 13:18:19.159157 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:19.170644 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 1.43799e+06us
I20260709 13:18:19.163950 18542 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.165160 18418 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:19.144522 16561 connection.cc:256] RPC call timeout handler was delayed by -1.7989493640000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23670000000000002s for a response
I20260709 13:18:19.154047 18434 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.118594 18340 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:19.098837 18543 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.135910 17415 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
W20260709 13:18:19.172240 16561 connection.cc:256] RPC call timeout handler was delayed by -1.6706403359999058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3168s for a response
W20260709 13:18:19.172964 16561 connection.cc:256] RPC call timeout handler was delayed by -1.5658398020000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 13:18:19.173309 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4731398020001052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 13:18:19.173629 16561 connection.cc:256] RPC call timeout handler was delayed by -1.3246398020000925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
W20260709 13:18:19.173949 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1473398020000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:18:19.174188 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1428398020000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
W20260709 13:18:19.174443 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0906398019999415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 13:18:19.169143 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.166585 18177 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.178684 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.179936 16744 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.187223 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:19.130208 18431 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:18:19.191573 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.224807 16662 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:19.159196 18421 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:19.225404 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 134513us
W20260709 13:18:19.231096 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.183386 18477 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.232100 18477 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.225545 16737 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:19.175256 18418 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.172199 18556 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.228163 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.239954 16741 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.176591 18300 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:19.247975 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.253186 16744 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:19.235464 18192 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:19.261411 16739 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:19.237193 18421 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:19.263392 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.264281 16737 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:19.241822 18474 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:19.268791 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 81661.7us
I20260709 13:18:19.270089 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.270893 16737 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:19.249547 18477 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:19.236361 18177 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:18:19.273780 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:19.256768 18297 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:19.276645 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.279150 16737 raft_consensus.cc:2410] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:19.259191 18431 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:19.287062 16737 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.284842 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.257453 18556 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:18:19.260455 18235 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:18:19.263335 18343 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:18:19.279863 18474 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:19.282011 18297 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.290635 18501 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:19.293478 18431 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.272794 18535 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:18:19.272547 18559 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:18:19.307526 16737 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:19.306437 18365 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:19.338267 18365 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:19.338742 18365 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.339854 18383 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:19.340915 18544 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:19.343928 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:19.334419 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55617.2us
I20260709 13:18:19.308106 18474 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:19.319619 18297 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:19.351255 18583 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:18:19.354058 18240 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:19.358990 18567 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:19.359426 18425 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.306712 18527 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:18:19.359817 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:19.390400 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.368702 18543 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:19.369371 18434 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.318617 18501 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.312798 18443 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:19.376653 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 231785us
I20260709 13:18:19.275657 18558 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.379916 16737 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.422389 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.337443 18553 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:19.361222 16590 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.382207 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:19.415164 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:19.422978 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 65955.7us
I20260709 13:18:19.440269 16593 raft_consensus.cc:2410] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:19.331146 18431 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.377115 18365 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.423765 18474 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.424455 18425 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.331544 18300 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:19.426127 18585 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:18:19.429823 18450 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.318428 18421 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:19.431365 18584 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:18:19.431284 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:19.377643 18559 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.432641 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:19.433648 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2108) took 2131 ms. Trace:
I20260709 13:18:19.441484 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:19.443307 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.445708 18543 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:19.444918 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:19.459883 16593 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.381680 18240 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:19.436442 18501 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:19.451821 17415 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.381162 18535 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:18:19.384356 18542 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.457387 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.445895 18546 raft_consensus.cc:2764] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:19.456137 16596 rpcz_store.cc:276] 0709 13:18:17.289399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.304451 (+ 15052us) service_pool.cc:224] Handling call
0709 13:18:17.308885 (+  4434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:17.309116 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:17.309148 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:17.309643 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 13:18:17.395428 (+ 85785us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:17.395458 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:17.395517 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:17.984509 (+588992us) raft_consensus.cc:1685] finished
0709 13:18:17.984536 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:19.382594 (+1398058us) spinlock_profiling.cc:243] Waited 4.02 ms on lock 0x7b54001fc2d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:19.420391 (+ 37797us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9246464,"thread_start_us":85375,"threads_started":1,"wal-append.queue_time_us":179846,"wal-append.run_cpu_time_us":3571,"wal-append.run_wall_time_us":1511416}
W20260709 13:18:19.460903 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 101143us
I20260709 13:18:19.462821 16742 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:19.467782 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:19.434726 18418 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.465211 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.459681 18474 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.469439 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:19.448278 18434 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:18:19.461550 18491 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.448990 18556 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:18:19.479285 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.480316 17414 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:18:19.481936 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:19.482219 18530 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.456720 18454 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:18:19.483640 18193 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:19.480316 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.518110 16739 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.459060 18559 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.469616 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:19.505609 17415 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:19.484898 18543 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:19.513625 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 70445.1us
I20260709 13:18:19.510248 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.520448 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:19.512274 18425 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:19.529621 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 77074.4us
I20260709 13:18:19.513008 18546 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:19.534093 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:19.535807 16739 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:19.546479 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.547868 16593 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:19.539793 16740 raft_consensus.cc:2368] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3.
I20260709 13:18:19.550675 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:19.532955 18193 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:19.529299 18542 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:19.535526 18546 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:19.572466 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.580440 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:19.581655 16739 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:19.550261 18592 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.584517 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:19.579514 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.587142 16739 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.570832 18022 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:19.589689 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:19.572062 18425 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:19.583170 18546 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:19.592999 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.573439 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:19.585955 18297 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.594023 18193 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.572817 18591 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:19.573807 18418 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.571863 18528 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:19.573015 18365 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.601236 16739 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:19.603222 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:19.585132 18542 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:19.587946 18556 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:19.590936 18022 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:19.618144 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 68693.4us
I20260709 13:18:19.618593 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:19.619109 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:19.619292 16665 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:19.621884 18599 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:18:19.623528 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:19.624442 16667 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:19.625424 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.627228 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.631989 18559 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:19.632586 18297 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.632678 18340 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:19.633868 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.610338 18605 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:19.634939 16671 raft_consensus.cc:2410] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:18:19.601590 18425 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:19.635325 18605 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:19.609030 18528 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:19.627235 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:19.637225 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:19.591305 18300 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.637631 16666 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:19.626642 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.591602 18597 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:18:19.635589 18425 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:19.706710 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.699891 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.683122 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.664402 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.665140 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.682331 16666 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
W20260709 13:18:19.723064 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.724085 16666 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:19.620435 18450 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:19.643515 18596 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:18:19.733476 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.616294 18543 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:19.726271 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 107236us
I20260709 13:18:19.727831 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.724143 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.729252 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.735302 17419 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.730212 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:19.618796 18022 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.675195 18365 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:19.704690 18530 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:19.643451 18300 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:19.830775 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 86533.6us
W20260709 13:18:19.864785 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:19.640278 18591 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:19.836524 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51262.3us
I20260709 13:18:19.643464 18424 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:19.693678 18434 raft_consensus.cc:476] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Not starting pre-election -- already a leader
I20260709 13:18:19.830080 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:19.879159 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.879592 16669 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:19.879755 18365 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:19.730823 18421 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:19.886371 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55797.1us
I20260709 13:18:19.855052 18193 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:19.856252 18491 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:19.949460 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:19.881085 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:19.829860 18543 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:19.729895 18535 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.203s
I20260709 13:18:19.957692 16669 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.705268 18528 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:19.943161 17411 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:19.739394 18253 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.223s
I20260709 13:18:19.944922 18193 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:19.855610 18549 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:18:19.984804 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.987879 18365 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:19.988888 17412 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.992063 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.993562 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:19.994787 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:19.995055 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:19.995452 18604 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:18:19.995854 16669 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:19.996167 16671 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:19.996217 18425 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:20.017241 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:20.024519 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 142266us
I20260709 13:18:19.975778 18606 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:19.987640 18431 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 13:18:19.980679 18553 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:19.984714 18193 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:19.885103 18542 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.037854 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:19.985818 18597 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:20.038688 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.038725 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.045221 16669 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:19.957203 18611 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:18:20.041599 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:20.062893 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:19.974555 18177 consensus_queue.cc:1175] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 2 last_received_current_leader { term: 3 index: 3 } }
I20260709 13:18:20.069846 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:19.966980 18605 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.077818 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:20.080354 16742 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.035207 18609 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:18:20.043190 18597 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:20.084157 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:20.044842 18559 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:18:20.046355 18553 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.041006 18431 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.088610 16669 raft_consensus.cc:2376] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4. Re-sending same reply.
I20260709 13:18:20.073062 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:20.105067 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:20.098136 18501 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.631s	user 0.010s	sys 0.000s
I20260709 13:18:20.113217 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.114002 16669 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:20.114897 18609 raft_consensus.cc:2764] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:18:20.116796 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:20.120644 16740 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:20.121124 18609 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:20.095709 18559 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:20.122658 16742 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:20.125020 16665 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:20.107818 18615 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:18:20.097335 18553 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.126139 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:20.095335 18300 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:20.105818 18253 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.086946 18614 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.138378 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.059578 18591 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.141758 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.111701 17411 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.147696 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:20.126971 18592 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.002s
I20260709 13:18:20.121661 18609 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.155251 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.155849 16669 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:20.084273 18606 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:18:20.156838 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.125105 18559 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:20.160106 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1646) took 1396 ms. Trace:
I20260709 13:18:20.161111 16664 rpcz_store.cc:276] 0709 13:18:18.763825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.833771 (+ 69946us) service_pool.cc:224] Handling call
0709 13:18:20.159913 (+1326142us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b540009db54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:20.160049 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21075328}
I20260709 13:18:20.162460 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.163394 16664 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:20.164814 16664 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:20.160259 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.166055 16745 raft_consensus.cc:2376] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5. Re-sending same reply.
I20260709 13:18:20.168215 18431 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:20.148725 18611 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.169361 18611 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.171695 18611 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.138883 18597 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.173228 18597 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:20.173269 18193 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.175079 17411 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
W20260709 13:18:20.148079 18546 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
W20260709 13:18:20.181586 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64857.9us
I20260709 13:18:20.125348 18501 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.187664 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:20.193053 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:20.195266 18616 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.196209 18239 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:20.195853 18616 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:20.196653 18239 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:20.197672 18239 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.158149 18592 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.199687 16593 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.158419 18609 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.200659 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.202557 16595 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:20.205444 16742 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:20.216493 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.217518 16742 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:20.219213 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:20.219436 16589 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:20.219843 16737 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.220377 18501 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.149261 18253 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:20.225164 18598 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.711s	user 0.006s	sys 0.004s
I20260709 13:18:20.218712 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.227684 16742 raft_consensus.cc:2410] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:20.179581 18559 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:20.198462 18616 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.235512 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.191262 18606 consensus_queue.cc:1243] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:20.240401 18343 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:20.235632 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:20.244045 18022 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.245316 16671 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:20.236454 18239 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:20.207705 18605 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:20.187088 18546 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.250272 18546 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.237593 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:20.238474 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:20.244787 18240 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.251168 17415 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.251422 18240 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.252641 18253 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.253783 16744 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:20.236197 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.255227 16744 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:20.255285 16587 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:20.256516 16744 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:20.258325 18421 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:20.259095 18421 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:20.259996 18421 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:20.282084 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 94540.2us
I20260709 13:18:20.268549 16663 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:20.288085 16737 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
W20260709 13:18:20.298594 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61071.1us
I20260709 13:18:20.295392 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.300758 17376 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:20.301527 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:20.303870 18022 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:20.284113 16739 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:20.305977 17415 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
W20260709 13:18:20.306463 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.298322 18559 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.307468 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:20.308292 18474 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:20.308980 18474 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:20.310122 18474 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.317440 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:20.318773 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:20.298489 18600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.001s	sys 0.019s
I20260709 13:18:20.321516 16745 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:20.322790 16736 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:20.298188 17411 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:20.323596 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.297503 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 411) took 1182 ms. Trace:
I20260709 13:18:20.328019 18193 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:20.314061 18598 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:20.322058 18618 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:20.339520 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.347455 16592 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:20.339957 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1645) took 1576 ms. Trace:
I20260709 13:18:20.343605 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:20.332789 18622 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:20.355767 16592 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:20.355942 16670 rpcz_store.cc:276] 0709 13:18:18.763604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:18.828570 (+ 64966us) service_pool.cc:224] Handling call
0709 13:18:20.339684 (+1511114us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b540033b654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:20.339870 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919592320}
I20260709 13:18:20.323235 18559 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:20.347716 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:20.319018 18553 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:20.334631 18600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.002s	sys 0.019s
I20260709 13:18:20.359050 16561 leader_election.cc:400] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
I20260709 13:18:20.370271 16561 leader_election.cc:403] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:20.363587 16636 leader_election.cc:400] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 2 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3.
I20260709 13:18:20.363543 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:20.358294 18622 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.366541 16592 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.381345 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:20.345702 18618 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.386307 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:20.366158 18559 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.369174 18491 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:20.388434 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:20.369319 18553 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.367710 18605 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.389225 18618 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.388566 17409 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.390978 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:20.379745 18622 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:20.388685 18501 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.394634 18431 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.415069 16594 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.419149 18474 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.421056 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.437381 16744 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:20.427011 18249 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:20.343750 17414 rpcz_store.cc:276] 0709 13:18:19.115397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.121374 (+  5977us) service_pool.cc:224] Handling call
0709 13:18:19.121745 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:19.511311 (+389566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.297181 (+785870us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:20.297404 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773400448}
I20260709 13:18:20.388944 18238 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:20.408731 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:20.444344 16743 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 1.
I20260709 13:18:20.445603 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:20.432004 16587 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:20.432865 16589 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:20.447715 16743 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:20.390390 18614 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:20.412252 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:20.399010 18559 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:20.438753 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:20.440388 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52025.6us
I20260709 13:18:20.402228 18553 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:20.406217 18491 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:20.452220 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:20.454033 16743 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.454559 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:20.455961 16708 leader_election.cc:336] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.516s (SENT)
I20260709 13:18:20.441370 18239 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:20.457948 18249 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:20.479032 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 67736.2us
W20260709 13:18:20.458155 18628 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:20.476402 16743 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:20.479507 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:20.480289 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.448755 18614 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.490707 16744 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:20.489367 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.480690 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:20.479791 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:20.443485 18474 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.442606 18431 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.481562 18606 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:20.477268 18239 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.490463 18600 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:20.507102 18365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.011s	sys 0.007s
W20260709 13:18:20.625022 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 145298us
W20260709 13:18:20.625828 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 137259us
I20260709 13:18:20.502539 18363 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:20.498601 18614 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.500936 18547 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.629292 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:20.536166 18592 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:20.570188 18606 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.631603 18600 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.617568 18239 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.657689 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.497781 18591 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:18:20.621928 18477 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:20.666807 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:20.621878 18365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.011s	sys 0.008s
I20260709 13:18:20.675940 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.629742 18421 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.682516 16588 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 1.
I20260709 13:18:20.631090 18363 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:20.688064 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 183419us
W20260709 13:18:20.705469 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76308.8us
I20260709 13:18:20.707979 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.708905 16594 raft_consensus.cc:2368] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:20.667913 18606 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:20.725849 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 269915us
I20260709 13:18:20.666250 18535 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:20.758114 16708 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.601s (SENT)
I20260709 13:18:20.758908 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:20.759445 16588 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:20.762307 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:20.763319 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57540.4us
I20260709 13:18:20.707324 18549 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:20.793150 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:20.796001 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.797118 16588 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:20.798452 18198 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:20.800400 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 94158.8us
I20260709 13:18:20.694810 18638 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:20.723867 18597 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:20.800284 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.673048 18592 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:20.809128 17374 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.770s (SENT)
W20260709 13:18:20.812251 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:20.760450 18535 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.818722 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:20.799189 18198 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.636844 18491 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.823071 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:20.850359 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49072.5us
I20260709 13:18:20.711122 18609 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.821518 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.710624 18421 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.858376 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:20.859097 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57455.4us
I20260709 13:18:20.664923 18249 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:20.862264 16595 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:20.865270 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:20.820272 18022 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:20.869024 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.809214 18638 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:20.868268 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.764478 18365 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:18:20.802402 18527 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:18:20.816578 18549 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.850135 18613 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:18:20.869657 16594 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:20.980913 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 181) took 1599 ms. Trace:
I20260709 13:18:20.906288 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:20.980997 16588 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:20.856271 18535 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:21.267666 16560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.000s	sys 0.011s
W20260709 13:18:21.365538 16560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.002s	sys 0.013s
I20260709 13:18:20.979342 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:21.359272 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.362224 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.103284 18474 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:21.386425 16587 raft_consensus.cc:2376] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4. Re-sending same reply.
I20260709 13:18:20.867410 18193 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.523s
I20260709 13:18:21.387876 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:21.008803 18637 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.519s	user 0.002s	sys 0.000s
I20260709 13:18:21.389038 16587 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 3: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:21.027702 18647 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:21.357681 16738 rpcz_store.cc:276] 0709 13:18:19.381187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.244270 (+863083us) service_pool.cc:224] Handling call
0709 13:18:20.244610 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.298841 (+ 54231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.298954 (+   113us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.320964 (+ 22010us) log.cc:844] Serialized 26 byte log entry
0709 13:18:20.528072 (+207108us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:20.528136 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.528247 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:20.980667 (+452420us) raft_consensus.cc:1685] finished
0709 13:18:20.980691 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:20.980844 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13527,"raft.queue_time_us":54814,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":17155,"thread_start_us":151717,"threads_started":1,"wal-append.queue_time_us":500200}
I20260709 13:18:21.390347 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:21.391873 16736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.005s	sys 0.011s
W20260709 13:18:21.392545 16736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.005s	sys 0.011s
I20260709 13:18:21.393100 18671 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.202702 18559 consensus_meta.cc:287] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.726s	user 0.006s	sys 0.000s
I20260709 13:18:21.394495 18559 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.371171 16634 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:21.394766 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.276093 18656 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.397351 18559 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:21.397397 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.400468 16587 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:21.302520 17409 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.001s
I20260709 13:18:21.401207 17409 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
W20260709 13:18:21.204502 18238 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.756s	user 0.007s	sys 0.003s
W20260709 13:18:21.337172 18632 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.569s	user 0.001s	sys 0.000s
I20260709 13:18:21.404569 18238 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:21.405910 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 371) took 1043 ms. Trace:
I20260709 13:18:20.858533 18646 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.406494 17409 rpcz_store.cc:276] 0709 13:18:20.362852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.363081 (+   229us) service_pool.cc:224] Handling call
0709 13:18:21.405819 (+1042738us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:21.101635 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 343611us
I20260709 13:18:20.937366 18454 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.446s
I20260709 13:18:21.372576 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.409785 16594 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:21.410825 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.274943 18652 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.207123 18633 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.646s	user 0.001s	sys 0.000s
I20260709 13:18:21.412132 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.412783 18652 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:20.869414 18022 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:20.884433 18363 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.413934 18022 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.414152 18363 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.362226 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.393536 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2105) took 1035 ms. Trace:
I20260709 13:18:21.415892 16736 rpcz_store.cc:276] 0709 13:18:20.357831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.490368 (+132537us) service_pool.cc:224] Handling call
0709 13:18:20.490715 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.497436 (+  6721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.497466 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.497936 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:18:21.393079 (+895143us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:21.393138 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.393228 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.393282 (+    54us) raft_consensus.cc:1685] finished
0709 13:18:21.393300 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.393468 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1111,"raft.queue_time_us":4757,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":6366,"thread_start_us":736292,"threads_started":1,"wal-append.queue_time_us":806921}
I20260709 13:18:21.397351 18656 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.419744 16743 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:21.422782 18022 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.425194 18022 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:21.394480 18671 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.413275 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:20.873590 18638 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:21.407011 18646 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.401484 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.365186 18198 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:20.938021 18565 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.519s
I20260709 13:18:21.359272 16591 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:20.863323 18600 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:21.428395 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69211.7us
W20260709 13:18:21.359141 18239 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.684s	user 0.012s	sys 0.000s
W20260709 13:18:21.198402 18631 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.545s	user 0.000s	sys 0.001s
W20260709 13:18:21.181506 18644 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:18:20.964651 18640 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:21.435382 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.436697 18652 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:21.439636 16711 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.026s	sys 0.024s
I20260709 13:18:21.224218 18424 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:18:21.355578 16740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 177) took 1980 ms (client timeout 1774 ms). Trace:
I20260709 13:18:21.421794 18656 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:21.690608 16596 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:18:21.691427 16711 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.029s	sys 0.030s
I20260709 13:18:21.691852 16594 raft_consensus.cc:2376] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4. Re-sending same reply.
I20260709 13:18:21.692953 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:21.694536 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.698904 16591 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:21.705423 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:21.543864 18553 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.058s	user 0.010s	sys 0.000s
W20260709 13:18:21.707640 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 297962us
W20260709 13:18:21.685981 18501 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.250s	user 0.009s	sys 0.002s
W20260709 13:18:21.712869 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.624467 18491 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.013s	sys 0.020s
W20260709 13:18:21.454116 18634 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.788s	user 0.001s	sys 0.000s
W20260709 13:18:21.545382 18581 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.011s	sys 0.009s
W20260709 13:18:21.717264 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 331709us
W20260709 13:18:21.718417 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 347330us
W20260709 13:18:21.723634 18581 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.017s	sys 0.012s
I20260709 13:18:21.498137 18447 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:21.725571 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:21.448067 18647 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:21.726193 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.691489 18559 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:21.712711 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.545s (SENT)
W20260709 13:18:21.706499 18605 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.295s	user 0.012s	sys 0.001s
W20260709 13:18:21.712977 16562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.003s	sys 0.010s
W20260709 13:18:21.763762 16562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.004s	sys 0.010s
I20260709 13:18:21.448632 18421 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:21.722787 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.526089 18609 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:21.438184 18193 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:21.438632 18657 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:21.710573 18198 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:21.718660 18491 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.021s	sys 0.021s
W20260709 13:18:21.759859 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.000s	sys 0.041s
W20260709 13:18:21.775153 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.000s	sys 0.041s
I20260709 13:18:21.727276 16596 raft_consensus.cc:2376] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4. Re-sending same reply.
I20260709 13:18:21.767426 16591 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:21.776371 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.773191 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.776980 16591 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:21.488729 18249 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.272s
W20260709 13:18:21.759008 18645 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 13:18:21.761525 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:21.736992 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 43804us
I20260709 13:18:21.728068 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.785027 16594 raft_consensus.cc:2368] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:21.729564 18447 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.705191 18239 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.779333 18654 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:18:21.784376 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.791224 16591 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:21.790313 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 981178us
I20260709 13:18:21.720906 18501 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4.
W20260709 13:18:21.689774 16740 rpcz_store.cc:269] 0709 13:18:19.374831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.573083 (+198252us) service_pool.cc:224] Handling call
0709 13:18:21.355262 (+1782179us) spinlock_profiling.cc:243] Waited 92.5 ms on lock 0x7b54003d4354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:21.355452 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212699008}
I20260709 13:18:21.769423 18605 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:21.786125 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73435.1us
W20260709 13:18:21.795086 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.791s (SENT)
W20260709 13:18:21.800835 16708 leader_election.cc:336] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.747s (SENT)
I20260709 13:18:21.801152 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:21.790095 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.787972 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:21.796223 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.803700 16594 raft_consensus.cc:2410] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:21.801282 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:21.720695 18553 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.809895 18666 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:21.810316 18618 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.374s	user 0.011s	sys 0.001s
I20260709 13:18:21.767217 18559 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.813454 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.786319 18447 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:21.790577 18656 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:21.815110 18685 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.815819 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.817461 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2100) took 1733 ms. Trace:
I20260709 13:18:21.818588 16594 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:21.810678 18666 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.702762 18640 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:21.818701 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:21.781600 18549 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:21.812186 16589 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:21.820216 16596 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.833190 16741 rpcz_store.cc:276] 0709 13:18:20.083959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.352443 (+268484us) service_pool.cc:224] Handling call
0709 13:18:20.352790 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.353176 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.353202 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.353642 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:18:20.452934 (+ 99292us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:20.452984 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.453048 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.453138 (+1000090us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:21.807639 (+354501us) raft_consensus.cc:1685] finished
0709 13:18:21.807662 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.817392 (+  9730us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5148,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":7186,"spinlock_wait_cycles":12130560,"thread_start_us":98930,"threads_started":1,"wal-append.queue_time_us":149644}
I20260709 13:18:21.773945 18193 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.816520 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 368) took 1463 ms. Trace:
I20260709 13:18:21.838261 18658 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:21.812449 18542 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:21.811904 18581 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:21.825295 18622 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.388s	user 0.010s	sys 0.002s
I20260709 13:18:21.823470 18421 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:21.789007 18198 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.835772 18657 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:21.855841 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55027.2us
I20260709 13:18:21.858804 18617 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:21.859323 18617 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:21.811847 18618 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:21.867208 18614 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.210s	user 0.007s	sys 0.006s
I20260709 13:18:21.867746 18614 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.854846 18249 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:21.849697 18581 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:21.877701 16591 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:21.855190 18658 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:21.883296 16739 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2082) took 2269 ms (client timeout 1630 ms). Trace:
I20260709 13:18:21.886432 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:21.889592 16739 rpcz_store.cc:269] 0709 13:18:19.603240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.636829 (+ 33589us) service_pool.cc:224] Handling call
0709 13:18:21.868616 (+2231787us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400049554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:21.872925 (+  4309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363822464}
I20260709 13:18:21.889962 16591 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:21.891767 16594 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:21.892735 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:21.893433 16594 raft_consensus.cc:2368] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3.
I20260709 13:18:21.894125 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.894661 16594 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:21.896064 16736 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:21.897174 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:21.897864 16744 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:21.898357 16593 raft_consensus.cc:2368] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:21.903255 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:21.905609 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.906092 16737 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:21.906508 16594 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:21.907507 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:21.908208 16739 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:21.911049 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.847899 18685 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.914991 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:21.915758 16744 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:21.916774 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:21.889875 18581 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:21.852288 18542 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:21.891278 18547 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:18:21.921535 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2095) took 2218 ms. Trace:
I20260709 13:18:21.923450 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:21.925608 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:21.926995 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:21.919266 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.976431 16591 raft_consensus.cc:2368] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:21.977633 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.847711 17411 rpcz_store.cc:276] 0709 13:18:20.353288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.353435 (+   147us) service_pool.cc:224] Handling call
0709 13:18:20.353793 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.367759 (+ 13966us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.370363 (+  2604us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.430579 (+ 60216us) log.cc:844] Serialized 26 byte log entry
0709 13:18:20.632631 (+202052us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:20.735246 (+102615us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.735600 (+   354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.735699 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.816275 (+1080576us) raft_consensus.cc:1685] finished
0709 13:18:21.816305 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.816453 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":355522,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":68,"raft.queue_time_us":2021,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":4425,"thread_start_us":292871,"threads_started":2,"wal-append.queue_time_us":422245}
I20260709 13:18:21.924376 18527 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:21.855182 18622 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:21.970268 16742 rpcz_store.cc:276] 0709 13:18:19.703368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.276689 (+573321us) service_pool.cc:224] Handling call
0709 13:18:20.277159 (+   470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:20.298925 (+ 21766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.298954 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:20.315172 (+ 16218us) log.cc:844] Serialized 52 byte log entry
0709 13:18:20.375268 (+ 60096us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:20.451309 (+ 76041us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.452075 (+   766us) log.cc:844] Serialized 14 byte log entry
0709 13:18:20.452509 (+   434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:20.452583 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:21.921310 (+1468727us) raft_consensus.cc:1685] finished
0709 13:18:21.921330 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:21.921474 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21273,"prepare.queue_time_us":78112,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":77,"raft.queue_time_us":22579,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":150,"spinlock_wait_cycles":11776640,"thread_start_us":95247,"threads_started":2,"wal-append.queue_time_us":167552}
I20260709 13:18:21.947338 17419 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:21.996429 16595 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:21.951117 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:21.960675 17415 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:21.920100 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:21.927361 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:21.859047 18647 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.021844 17416 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:22.022393 16736 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.000443 16737 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:21.934669 18646 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:22.024369 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:21.922158 18363 consensus_meta.cc:287] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.002s
I20260709 13:18:21.924582 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.963338 18542 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:21.965344 18671 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:21.960237 17412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 362) took 2254 ms (client timeout 1889 ms). Trace:
I20260709 13:18:21.936728 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:21.958614 18660 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:18:22.056221 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:21.959729 18618 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.029667 18666 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:21.958546 18680 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.069703 16670 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:18:21.940722 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.071069 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.040143 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.022291 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.071966 16591 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:22.072839 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.034735 18434 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.037806 17412 rpcz_store.cc:269] 0709 13:18:19.705920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.708858 (+  2938us) service_pool.cc:224] Handling call
0709 13:18:21.959952 (+2251094us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b5400571054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:21.960123 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65057536}
I20260709 13:18:22.077613 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.078199 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.078395 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.079376 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:22.046540 17374 leader_election.cc:336] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.601s (SENT)
I20260709 13:18:22.047745 18553 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.045527 18657 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.050161 18606 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.288s	user 0.011s	sys 0.001s
I20260709 13:18:22.054684 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.027326 16741 raft_consensus.cc:2368] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5.
I20260709 13:18:22.061779 18640 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.063910 16738 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:22.066606 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.067852 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:22.075802 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:21.958249 18543 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:21.949545 18617 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.099882 16669 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:22.101516 16742 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.080832 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.081768 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.048892 18198 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:22.059072 16740 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.065913 18614 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:22.120934 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4455153719998179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 13:18:22.121564 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2673153719997572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
W20260709 13:18:22.121963 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2124153719998958s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
W20260709 13:18:22.122339 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1539153719998012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1889s for a response
I20260709 13:18:22.123625 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:22.108821 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62359.5us
I20260709 13:18:22.109663 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.125219 16591 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:22.126600 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:22.113999 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.128003 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.118095 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1647) took 1491 ms. Trace:
I20260709 13:18:22.131258 16662 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:22.138775 16589 rpcz_store.cc:276] 0709 13:18:20.626788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.636248 (+  9460us) service_pool.cc:224] Handling call
0709 13:18:22.117831 (+1481583us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540010e854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:22.117994 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2598557952}
I20260709 13:18:22.083820 18363 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.150362 16591 raft_consensus.cc:2410] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:18:22.154767 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.156190 16669 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:21.926277 18681 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.193s
I20260709 13:18:22.107789 17411 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Advancing to term 4
I20260709 13:18:22.161345 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:22.162701 16591 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:22.106993 18542 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.136907 18546 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.137907 18617 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:22.183585 16738 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:22.185171 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.085505 18434 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.139140 18640 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.188011 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.188822 16736 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.141043 18618 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:22.196727 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 73242.3us
I20260709 13:18:22.159096 18193 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.087925 18680 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.230012 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:22.231441 16589 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:22.098315 18606 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.164683 18658 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.079869 18638 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.268003 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:22.229641 18682 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
I20260709 13:18:22.107580 18447 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.281298 16589 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.314240 16596 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:22.314767 16742 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:22.316126 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48200.4us
I20260709 13:18:22.342535 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:22.160807 18363 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.351770 16594 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:22.310447 18641 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 1.202s	user 0.005s	sys 0.007s
I20260709 13:18:22.281932 18558 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:22.195035 18546 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.315222 18193 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.311677 18685 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.281364 18613 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:22.161295 18657 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.229274 18542 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.265161 18640 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:22.365516 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 250272us
I20260709 13:18:22.342489 18658 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.364611 18686 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:22.361666 18656 consensus_meta.cc:287] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.002s
W20260709 13:18:22.363849 18535 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.982s	user 0.009s	sys 0.004s
I20260709 13:18:22.364082 16587 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:22.377147 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:22.340983 18679 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:22.364415 18558 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.385005 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.375699 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 370) took 1989 ms. Trace:
I20260709 13:18:22.390957 16667 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:22.374061 18622 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.392591 16666 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:22.373917 18613 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:22.377800 18685 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.395661 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.396164 18681 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.397948 18640 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:22.398931 18669 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
W20260709 13:18:22.399549 18684 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:22.400502 18543 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.400952 18666 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.401335 18477 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.401962 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.403528 18177 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:22.403964 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.405109 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:22.386457 18654 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.385833 18679 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.390285 18558 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.408331 18434 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.408977 18559 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.406013 16592 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.384647 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 421) took 1511 ms. Trace:
I20260709 13:18:22.387072 18656 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:22.376819 18193 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.481768 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.486796 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.491562 16740 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.414289 18685 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:22.496524 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 137623us
I20260709 13:18:22.497189 18527 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:22.502641 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 372896us
W20260709 13:18:22.485603 18682 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:18:22.391574 18535 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.512161 16671 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:22.382099 18658 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:22.485118 17412 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:22.484426 18679 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.518934 18685 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.490342 18177 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:22.421411 18681 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.478833 18477 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.541009 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.541913 18670 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.542447 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:22.544327 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:22.426013 18543 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:22.544921 16739 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.543646 16664 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:22.543624 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.546550 16740 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.549047 16592 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:22.549710 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2145) took 3175 ms. Trace:
I20260709 13:18:22.550072 16592 rpcz_store.cc:276] 0709 13:18:19.374580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.376115 (+  1535us) service_pool.cc:224] Handling call
0709 13:18:19.376435 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:20.347349 (+970914us) spinlock_profiling.cc:243] Waited 50.7 ms on lock 0x7b54002ac4a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb5bdc0 00007f327fbe61c0 00007f327fbe7231 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c
0709 13:18:22.405559 (+2058210us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b58004c0ad0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:18:22.549468 (+143909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.549661 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4137501824}
I20260709 13:18:22.551462 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.552052 16592 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.552327 16738 raft_consensus.cc:2410] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:22.552349 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.485759 18666 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:22.553079 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.505653 18686 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:18:22.554812 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.555677 18691 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:22.559187 16742 raft_consensus.cc:2376] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5. Re-sending same reply.
I20260709 13:18:22.494786 18654 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.562412 16740 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:22.556191 18567 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:18:22.555228 18177 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.565680 18558 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.566431 18693 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.392261 17410 rpcz_store.cc:276] 0709 13:18:20.353939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.366977 (+ 13038us) service_pool.cc:224] Handling call
0709 13:18:20.407981 (+ 41004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:20.484076 (+ 76095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.496538 (+ 12462us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:20.656786 (+160248us) log.cc:844] Serialized 26 byte log entry
0709 13:18:20.988429 (+331643us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:21.009023 (+ 20594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:21.065993 (+ 56970us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:22.264392 (+1198399us) raft_consensus.cc:1685] finished
0709 13:18:22.266521 (+  2129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.341326 (+ 74805us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12121,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":9157,"thread_start_us":85522,"threads_started":1,"wal-append.queue_time_us":837588}
W20260709 13:18:22.566821 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 85158.8us
I20260709 13:18:22.612788 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:22.567451 18701 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:22.571022 18704 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.570633 18685 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.565567 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.583988 16671 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:22.584721 16592 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:22.587306 18647 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:22.588128 18681 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.589090 18657 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.627138 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.589457 18665 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.590360 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.591598 18477 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:22.589025 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.551625 18618 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:22.606896 16744 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.029s	sys 0.050s
I20260709 13:18:22.575124 18697 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.614274 16739 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.511480 18613 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.615002 18546 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:22.657279 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44650.3us
I20260709 13:18:22.624212 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:22.658140 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:22.612392 18687 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:18:22.556500 18698 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:22.645746 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:22.648591 16744 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.030s	sys 0.050s
I20260709 13:18:22.662336 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:22.655786 18710 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:22.623620 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.572872 18606 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.658329 16738 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:22.578315 18363 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.651134 18618 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:22.659793 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69521.3us
I20260709 13:18:22.653332 18542 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.612236 18656 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:18:22.671041 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:22.675993 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.680971 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.636798 18680 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:22.614593 18701 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:18:22.648881 18447 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.651845 18681 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:22.710427 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.689108 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.627966 18547 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.694897 16738 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:22.697491 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.493438 17414 rpcz_store.cc:276] 0709 13:18:20.873279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.885057 (+ 11778us) service_pool.cc:224] Handling call
0709 13:18:21.150292 (+265235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:21.241126 (+ 90834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:21.249476 (+  8350us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:21.319416 (+ 69940us) log.cc:844] Serialized 26 byte log entry
0709 13:18:21.762943 (+443527us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:22.148823 (+385880us) log.cc:844] Serialized 14 byte log entry
0709 13:18:22.149137 (+   314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.149266 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:22.373438 (+224172us) raft_consensus.cc:1685] finished
0709 13:18:22.375926 (+  2488us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:22.381892 (+  5966us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":511469,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":317732,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":121759,"thread_start_us":761282,"threads_started":2,"wal-append.queue_time_us":503393}
I20260709 13:18:22.690011 17410 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:22.675261 18698 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.705816 18708 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.674849 18613 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.618016 18693 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.713414 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.716634 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.718830 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:22.722764 17409 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:22.723016 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.682303 18697 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.723302 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.723932 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.724274 18558 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.679843 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.724543 18447 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:22.725355 18543 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:22.691205 18618 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.660811 18704 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.751472 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.751725 16737 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.664202 18665 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.754447 16738 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:22.755671 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:22.726917 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 64632.2us
I20260709 13:18:22.753753 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.760226 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:22.793547 16666 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.793280 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.797577 16737 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 13:18:22.763401 16665 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:22.801779 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48105.4us
I20260709 13:18:22.758545 18542 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:22.808364 18698 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.808627 18697 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.748500 18708 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:22.799379 18558 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:22.811030 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.812748 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.722200 18710 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:22.818547 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 420) took 2511 ms. Trace:
I20260709 13:18:22.818460 18704 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.733860 18559 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:18:22.764145 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.839347 16667 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:22.806421 18715 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:22.752593 18699 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:22.754827 18546 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:22.792220 18447 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:22.754812 18193 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.828159 17415 rpcz_store.cc:276] 0709 13:18:20.306828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.308332 (+  1504us) service_pool.cc:224] Handling call
0709 13:18:21.960317 (+1651985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.021124 (+ 60807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.818315 (+797191us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400571050. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:22.818459 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3798458240}
I20260709 13:18:22.660140 18527 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:22.763135 18535 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.840746 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.763010 18658 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.825510 18708 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:22.830747 18707 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:22.830257 16492 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:22.832152 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.830574 18542 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:18:22.830314 16561 leader_election.cc:400] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5.
I20260709 13:18:22.885593 16561 leader_election.cc:403] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:22.727177 18547 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.826830 18698 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:22.888542 18431 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:22.889237 18623 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:18:22.860532 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.851064 18546 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:22.836628 18666 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:22.851305 18654 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:22.886945 18617 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.656s	user 0.006s	sys 0.006s
I20260709 13:18:22.876847 18679 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.907999 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:22.908878 16596 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:22.908898 16738 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.883769 18685 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:22.886847 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 56697.8us
I20260709 13:18:22.917229 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:22.897259 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 1.04044e+06us
I20260709 13:18:22.869391 16665 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:22.900286 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.919512 16736 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:22.901778 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1661) took 1994 ms. Trace:
I20260709 13:18:22.920442 16595 rpcz_store.cc:276] 0709 13:18:20.907018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.907167 (+   149us) service_pool.cc:224] Handling call
0709 13:18:22.901622 (+1994455us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400052654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:22.901724 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1078926720}
I20260709 13:18:22.893123 18658 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5.
W20260709 13:18:22.907613 16590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2151) took 3531 ms (client timeout 2592 ms). Trace:
W20260709 13:18:22.885722 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 92564.7us
I20260709 13:18:22.921599 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.907132 18680 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:22.918792 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.940913 16667 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:22.893451 18715 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:22.892488 18546 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:22.947060 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:22.926810 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:22.928318 16590 rpcz_store.cc:269] 0709 13:18:19.376599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:19.452959 (+ 76360us) service_pool.cc:224] Handling call
0709 13:18:22.907352 (+3454393us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b5400157a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:22.907514 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5011100544}
I20260709 13:18:22.948550 18685 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:22.948599 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:22.948945 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:22.949290 18685 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:22.945923 18642 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 2.035s	user 0.004s	sys 0.012s
I20260709 13:18:22.951166 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.949594 16590 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.950958 18558 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:22.953979 18558 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:22.954327 18693 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.930138 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.955202 18558 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.951798 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:22.956342 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.852785 18699 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.947484 18680 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.952725 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.958178 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:22.958808 16737 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:22.959084 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.959824 16667 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:22.916083 18679 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:22.961062 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:22.961352 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:22.962514 18177 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:22.962148 18447 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:22.929276 18698 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:22.962185 16595 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:22.982607 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.983770 16662 raft_consensus.cc:2410] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:22.984783 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:22.986152 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:22.986291 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:22.987632 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:22.988701 17414 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:22.989898 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:22.992304 16708 connection.cc:256] RPC call timeout handler was delayed by -2.1470742590001919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2147s for a response
W20260709 13:18:22.992931 16708 connection.cc:256] RPC call timeout handler was delayed by -2.1317742590001671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 13:18:22.993353 16708 connection.cc:256] RPC call timeout handler was delayed by -2.0995041500002571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
W20260709 13:18:22.993705 16708 connection.cc:256] RPC call timeout handler was delayed by -2.0552742590000435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 13:18:22.994072 16708 connection.cc:256] RPC call timeout handler was delayed by -2.0547243060000255s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2109s for a response
W20260709 13:18:22.994414 16708 connection.cc:256] RPC call timeout handler was delayed by -2.0354742590002388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2271s for a response
I20260709 13:18:22.982379 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:22.961200 17411 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.779s	user 0.011s	sys 0.000s
I20260709 13:18:23.000458 18665 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:22.926190 18710 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:18:22.965540 18658 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:23.008460 18238 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.008770 18658 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.008986 18238 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:23.009248 18658 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:22.950347 18685 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:22.996924 16667 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
W20260709 13:18:22.994999 16708 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.821s (SENT)
W20260709 13:18:23.106093 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 111104us
W20260709 13:18:23.107167 16708 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.828s (SENT)
I20260709 13:18:22.991971 18698 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:23.128114 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:23.129393 16588 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
W20260709 13:18:23.129914 16588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1676) took 1860 ms (client timeout 1545 ms). Trace:
W20260709 13:18:23.130261 16588 rpcz_store.cc:269] 0709 13:18:21.269247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.360265 (+ 91018us) service_pool.cc:224] Handling call
0709 13:18:23.127885 (+1767620us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b5800010e98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:23.129848 (+  1963us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35737472}
I20260709 13:18:23.133754 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:23.140785 18720 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:23.141364 18720 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:23.141762 18720 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.151141 16671 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
W20260709 13:18:23.153642 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.009881 18238 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:23.019008 18618 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:23.172834 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:23.532889 16595 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:23.545603 16738 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.001s
W20260709 13:18:23.545756 17376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.000s	sys 0.010s
I20260709 13:18:23.124111 17411 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:18:23.257658 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:23.424957 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:23.546175 16738 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
W20260709 13:18:23.546888 17376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.000s	sys 0.010s
I20260709 13:18:23.553927 16737 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:23.417385 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 310237us
W20260709 13:18:23.555439 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.868s (SENT) [suppressed 35 similar messages]
W20260709 13:18:23.554317 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 520734us
I20260709 13:18:23.423468 18699 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:23.567638 18635 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 2.828s	user 0.019s	sys 0.034s
I20260709 13:18:23.569641 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:23.570379 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:23.571501 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 958) took 1514 ms. Trace:
I20260709 13:18:23.572105 16664 rpcz_store.cc:276] 0709 13:18:22.057293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.057430 (+   137us) service_pool.cc:224] Handling call
0709 13:18:22.079700 (+ 22270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.548609 (+468909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.571136 (+1022527us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b540033ac54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:23.571411 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1066543488}
I20260709 13:18:23.572710 16737 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:23.576483 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 167) took 1139 ms. Trace:
W20260709 13:18:23.609227 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88872us
I20260709 13:18:23.609484 16670 rpcz_store.cc:276] 0709 13:18:22.431366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.461583 (+ 30217us) service_pool.cc:224] Handling call
0709 13:18:23.569489 (+1107906us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:23.570807 (+  1318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3326208}
W20260709 13:18:23.326174 18717 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 13:18:23.454493 18666 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.619935 16588 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:23.639729 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2178) took 1863 ms. Trace:
I20260709 13:18:23.622177 18680 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:23.649034 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2176) took 1874 ms. Trace:
I20260709 13:18:23.649567 16596 rpcz_store.cc:276] 0709 13:18:21.775136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.343580 (+568444us) service_pool.cc:224] Handling call
0709 13:18:22.343860 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.909273 (+565413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.648730 (+739457us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54001faed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:23.648962 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299029888}
I20260709 13:18:23.627002 18527 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.562845 18528 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:23.629776 17412 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:18:23.552227 18581 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.616s	user 0.009s	sys 0.006s
I20260709 13:18:23.327132 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:23.644047 16588 rpcz_store.cc:276] 0709 13:18:21.775821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.560318 (+1784497us) service_pool.cc:224] Handling call
0709 13:18:23.619747 (+ 59429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.639393 (+ 19646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.639658 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.407958 16590 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:23.633853 18726 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:23.286674 18708 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:23.643432 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 243) took 1653 ms. Trace:
I20260709 13:18:23.674211 16588 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.676060 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2180) took 1899 ms. Trace:
I20260709 13:18:23.676478 16588 rpcz_store.cc:276] 0709 13:18:21.776417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.663198 (+1886781us) service_pool.cc:224] Handling call
0709 13:18:23.674053 (+ 10855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.675820 (+  1767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.676008 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.556576 18622 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:23.683215 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 184) took 2969 ms. Trace:
I20260709 13:18:23.683597 16745 rpcz_store.cc:276] 0709 13:18:20.706016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.706172 (+   156us) service_pool.cc:224] Handling call
0709 13:18:20.706513 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:20.766544 (+ 60031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:20.766590 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:20.766651 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:20.766697 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.675064 (+2908367us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b54002783d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:23.675115 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.675237 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137523968}
W20260709 13:18:23.667902 18447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.040s	sys 0.063s
I20260709 13:18:23.702374 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2181) took 1925 ms. Trace:
I20260709 13:18:23.155426 18715 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:23.673027 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 125) took 1112 ms. Trace:
I20260709 13:18:23.662180 18527 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:23.671375 17414 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 1.
I20260709 13:18:23.819865 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:23.820883 17414 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:23.709589 16588 rpcz_store.cc:276] 0709 13:18:21.776693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.677976 (+1901283us) service_pool.cc:224] Handling call
0709 13:18:23.701135 (+ 23159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.701334 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.701355 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.701394 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:23.701427 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.701477 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.701616 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.683195 18622 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:23.828084 18622 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:23.831478 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2182) took 2054 ms. Trace:
I20260709 13:18:23.402643 18698 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:23.712914 18546 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.747s	user 0.003s	sys 0.014s
I20260709 13:18:23.854733 18546 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:23.666395 18581 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:23.660431 18666 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:23.859066 18666 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:23.858428 18581 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:23.713495 18447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.040s	sys 0.063s
I20260709 13:18:23.751958 16595 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:23.839907 16737 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:23.841382 16738 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.677623 17411 rpcz_store.cc:276] 0709 13:18:21.989786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.025769 (+ 35983us) service_pool.cc:224] Handling call
0709 13:18:22.085092 (+ 59323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.560464 (+1475372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.640563 (+ 80099us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.849085 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.857501 16663 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:23.854540 16588 rpcz_store.cc:276] 0709 13:18:21.776940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.830169 (+2053229us) service_pool.cc:224] Handling call
0709 13:18:23.830686 (+   517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.830971 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.831013 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.831085 (+    72us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:23.831138 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.831218 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.831388 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.860954 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.837740 18719 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.871201 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:23.872172 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 316600us
W20260709 13:18:23.853289 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.017s	sys 0.043s
I20260709 13:18:23.873833 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2197) took 1315 ms. Trace:
I20260709 13:18:23.872329 16737 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:23.875038 16595 rpcz_store.cc:276] 0709 13:18:22.558021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.110659 (+552638us) service_pool.cc:224] Handling call
0709 13:18:23.873766 (+763107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.847121 18239 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:23.875962 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.017s	sys 0.043s
W20260709 13:18:23.876362 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 879836us
I20260709 13:18:23.877113 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:23.877940 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:23.878525 16744 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.876622 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:23.881294 16737 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:23.857211 18546 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:23.879958 16595 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:23.761801 16590 rpcz_store.cc:276] 0709 13:18:22.559201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.948646 (+389445us) service_pool.cc:224] Handling call
0709 13:18:23.671166 (+722520us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.880546 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:23.886817 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 276783us
I20260709 13:18:23.887369 16738 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:23.870826 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 305397us
I20260709 13:18:23.878562 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:23.884806 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2185) took 2107 ms. Trace:
W20260709 13:18:23.884445 18670 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.341s	user 0.015s	sys 0.000s
W20260709 13:18:23.892356 16708 leader_election.cc:336] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.921s (SENT)
I20260709 13:18:23.876434 18618 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:23.895694 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.901981 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2179) took 2125 ms. Trace:
W20260709 13:18:23.902668 17376 leader_election.cc:336] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.774s (SENT)
I20260709 13:18:23.903270 16595 rpcz_store.cc:276] 0709 13:18:21.777680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.879461 (+2101781us) service_pool.cc:224] Handling call
0709 13:18:23.879757 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.884243 (+  4486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.884750 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.906072 16738 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:23.904165 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:23.876838 18719 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:23.908267 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:23.909209 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.899852 18733 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.918735 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:23.920727 16738 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:23.909121 18720 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.767s	user 0.009s	sys 0.006s
I20260709 13:18:23.921852 18666 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.922205 18720 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:23.923750 18721 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:18:23.924551 18720 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:23.924975 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 429) took 1218 ms. Trace:
I20260709 13:18:23.925341 17415 rpcz_store.cc:276] 0709 13:18:22.706150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.893580 (+187430us) service_pool.cc:224] Handling call
0709 13:18:22.893973 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:22.894332 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.894360 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:22.917753 (+ 23393us) log.cc:844] Serialized 26 byte log entry
0709 13:18:23.226740 (+308987us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:23.226780 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.226884 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:23.924771 (+697887us) raft_consensus.cc:1685] finished
0709 13:18:23.924788 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.924917 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":5472,"thread_start_us":300806,"threads_started":1,"wal-append.queue_time_us":730970}
W20260709 13:18:23.930284 16708 connection.cc:256] RPC call timeout handler was delayed by -1.9409728230002656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 13:18:23.930735 16708 connection.cc:256] RPC call timeout handler was delayed by -1.938415495000072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 13:18:23.930850 16737 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:23.931089 16708 connection.cc:256] RPC call timeout handler was delayed by -1.9143443360001129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:18:23.931414 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8851728230001754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
I20260709 13:18:23.931933 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:23.932018 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8808041500001309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
I20260709 13:18:23.932399 16737 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:23.932670 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8334125480000694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17010000000000003s for a response
W20260709 13:18:23.933007 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8149728230002893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260709 13:18:23.933138 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:23.933320 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8149728230002893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260709 13:18:23.907253 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:23.935067 16590 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:23.935746 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2186) took 2157 ms. Trace:
I20260709 13:18:23.936149 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:23.936092 16590 rpcz_store.cc:276] 0709 13:18:21.777947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.901332 (+2123385us) service_pool.cc:224] Handling call
0709 13:18:23.918963 (+ 17631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.935476 (+ 16513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.935688 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.936633 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:23.937189 16739 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:23.937515 16590 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.938184 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2187) took 2160 ms. Trace:
I20260709 13:18:23.938226 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.938951 16736 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:23.939770 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 185) took 3053 ms. Trace:
I20260709 13:18:23.939851 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.940289 18719 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:23.940217 16743 rpcz_store.cc:276] 0709 13:18:20.886196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.886364 (+   168us) service_pool.cc:224] Handling call
0709 13:18:21.359614 (+473250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:21.420125 (+ 60511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.939556 (+2519431us) spinlock_profiling.cc:243] Waited 59.8 ms on lock 0x7b54003d0254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:23.939707 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":137436032}
I20260709 13:18:23.941110 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.941190 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:23.941435 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:23.941886 16736 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:23.941977 16743 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:23.933650 16737 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
W20260709 13:18:23.933656 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6891160060001766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27740000000000004s for a response
W20260709 13:18:23.947329 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6891160060001766s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27740000000000004s for a response
W20260709 13:18:23.947659 16708 connection.cc:256] RPC call timeout handler was delayed by -1.5035728230002405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
W20260709 13:18:23.947948 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4707125479999377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 13:18:23.948235 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4707125479999377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260709 13:18:23.948578 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4542160060000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30350000000000005s for a response
W20260709 13:18:23.948963 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4542160060000242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30350000000000005s for a response
W20260709 13:18:23.949345 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4416819089999535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1785s for a response
W20260709 13:18:23.949753 16708 connection.cc:256] RPC call timeout handler was delayed by -1.1248819090001234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
W20260709 13:18:23.951242 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48599.1us
I20260709 13:18:23.953692 16744 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:23.954658 16743 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:23.950132 16708 connection.cc:256] RPC call timeout handler was delayed by -1.1194819090001147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 13:18:23.955783 16708 connection.cc:256] RPC call timeout handler was delayed by -1.1194819090001147s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
I20260709 13:18:23.956869 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:23.944690 16634 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:23.959546 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:23.934660 17411 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.938617 16590 rpcz_store.cc:276] 0709 13:18:21.778197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.937028 (+2158831us) service_pool.cc:224] Handling call
0709 13:18:23.937337 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.937923 (+   586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.938135 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.962291 16590 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.962911 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2188) took 2184 ms. Trace:
I20260709 13:18:23.930331 18618 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:23.963541 18732 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:23.963246 16590 rpcz_store.cc:276] 0709 13:18:21.778451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.961766 (+2183315us) service_pool.cc:224] Handling call
0709 13:18:23.962112 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.962648 (+   536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.962830 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.964145 18732 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:23.964934 18658 consensus_meta.cc:287] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.955s	user 0.002s	sys 0.010s
I20260709 13:18:23.965420 16590 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:23.966167 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2189) took 2187 ms. Trace:
I20260709 13:18:23.909749 18670 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:23.974366 16738 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:23.978114 16739 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:23.979149 16738 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:23.966579 18732 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:23.939982 18544 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:23.935746 18453 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:23.989617 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:23.943696 18708 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:23.993501 16744 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:23.967914 18454 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:18:23.997193 18454 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:23.997423 18666 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:23.997584 18454 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:23.998051 18666 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:23.990492 18732 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:23.989004 18544 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:23.999552 18732 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:23.967767 18720 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:23.966464 16590 rpcz_store.cc:276] 0709 13:18:21.778656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.964786 (+2186130us) service_pool.cc:224] Handling call
0709 13:18:23.965174 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:23.965885 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.966116 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:23.999034 18666 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:23.965364 18658 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.008574 16739 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:23.983408 18731 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:23.967576 18581 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:18:24.003086 18670 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:23.992552 18453 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:24.032663 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:24.035526 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 156012us
I20260709 13:18:24.018363 16738 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:18:23.998564 18454 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:24.044939 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85497.5us
I20260709 13:18:24.036006 18527 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:24.036417 18670 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:24.026224 18544 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:24.056638 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86594.5us
I20260709 13:18:24.051559 18592 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:24.059422 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:24.062543 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:24.064379 16744 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
W20260709 13:18:24.066520 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 109731us
I20260709 13:18:24.046430 18581 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:24.044277 17414 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:24.027300 18731 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:24.056254 18454 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.068414 18544 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.069687 18670 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:24.070433 16668 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:24.070812 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 251) took 1081 ms. Trace:
I20260709 13:18:24.078068 18719 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.078611 18719 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.079020 18719 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.075881 18556 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.079811 18556 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:24.060829 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:24.082086 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.089622 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:24.065670 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:24.090934 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.069155 18581 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:24.100126 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:24.058071 17411 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:24.102340 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:24.104043 18592 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:24.105446 18592 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:24.105835 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.107172 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:24.111435 18592 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.113045 18592 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:24.113691 18592 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:24.114331 18592 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.045997 18732 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.068065 18708 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:18:24.058210 18453 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.119539 18611 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:24.119819 18742 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:24.119570 18733 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:24.121724 18528 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.076638 17414 rpcz_store.cc:276] 0709 13:18:22.989695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.819486 (+829791us) service_pool.cc:224] Handling call
0709 13:18:24.070732 (+251246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.123802 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 252) took 1133 ms. Trace:
I20260709 13:18:24.097782 18738 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:24.142544 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:24.158659 18235 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:24.162230 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2175) took 2388 ms. Trace:
I20260709 13:18:24.169739 16589 rpcz_store.cc:276] 0709 13:18:21.774229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.341888 (+567659us) service_pool.cc:224] Handling call
0709 13:18:22.342174 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:22.386685 (+ 44511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:22.386729 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:22.386785 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:22.386829 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.161847 (+1775018us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b54003584d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:24.161919 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.162134 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101835648}
I20260709 13:18:24.130340 18611 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.171809 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:24.181473 16662 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:24.133687 18740 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:24.140543 18528 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.135869 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 248) took 1147 ms. Trace:
I20260709 13:18:24.144277 18453 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:24.196007 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 93750.5us
I20260709 13:18:24.182539 18611 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:24.182108 18658 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:24.189182 17415 rpcz_store.cc:276] 0709 13:18:22.988845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.942104 (+953259us) service_pool.cc:224] Handling call
0709 13:18:23.942356 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.942552 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:24.041555 (+ 99003us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.135312 (+ 93757us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.135413 (+   101us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:24.135490 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.135591 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.135780 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135773,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":31751,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":2867,"spinlock_wait_cycles":7971584,"thread_start_us":191966,"threads_started":2,"wal-append.queue_time_us":125919}
I20260709 13:18:24.169409 18235 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.144345 17411 rpcz_store.cc:276] 0709 13:18:22.990028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.889395 (+899367us) service_pool.cc:224] Handling call
0709 13:18:24.121762 (+232367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.181435 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:24.188278 18528 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:24.197655 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51817.2us
I20260709 13:18:24.215716 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:24.217041 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:24.235852 18745 raft_consensus.cc:2764] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:18:24.215252 18235 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:24.196173 18740 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:24.217003 18693 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:18:24.241205 17415 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
W20260709 13:18:24.247393 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.241545 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:24.199182 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:24.198199 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:24.248744 16589 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:24.253573 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:24.254721 16589 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:24.198987 18715 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:18:24.255688 16590 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:24.182986 18581 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:24.250802 18739 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:18:24.255741 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:24.266916 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:24.267005 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 124544us
I20260709 13:18:24.271080 18744 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:24.271551 18744 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:24.271710 18708 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:24.272435 18744 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.276814 18581 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.287477 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2183) took 2510 ms. Trace:
I20260709 13:18:24.288384 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.287355 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:24.293076 16590 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:24.255827 18733 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:24.293092 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.298470 16589 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:24.302249 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2190) took 2493 ms. Trace:
I20260709 13:18:24.303121 16589 rpcz_store.cc:276] 0709 13:18:21.809158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.297793 (+2488635us) service_pool.cc:224] Handling call
0709 13:18:24.298145 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.301883 (+  3738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.302166 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.290072 16588 rpcz_store.cc:276] 0709 13:18:21.777174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.872288 (+2095114us) service_pool.cc:224] Handling call
0709 13:18:23.873304 (+  1016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:23.882231 (+  8927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:23.882263 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:23.884802 (+  2539us) log.cc:844] Serialized 26 byte log entry
0709 13:18:24.034054 (+149252us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:24.034096 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.034181 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:24.278266 (+244085us) raft_consensus.cc:1685] finished
0709 13:18:24.278288 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.287403 (+  9115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8547,"raft.queue_time_us":9617,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":15231,"thread_start_us":115683,"threads_started":1,"wal-append.queue_time_us":220396}
I20260709 13:18:24.263939 18742 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:24.305402 18748 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:18:24.307555 18715 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:24.260123 18673 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:18:24.290465 18744 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:24.315589 17411 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:18:24.316437 18749 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:24.260576 18750 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:18:24.297403 18745 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.320008 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:24.276957 18719 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.294091 18731 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:24.389415 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 69518.4us
I20260709 13:18:24.390565 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:24.328899 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 119) took 2472 ms. Trace:
W20260709 13:18:24.330107 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 88541us
I20260709 13:18:24.391561 16588 rpcz_store.cc:276] 0709 13:18:21.856114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.313894 (+2457780us) service_pool.cc:224] Handling call
0709 13:18:24.328137 (+ 14243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.328403 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.328445 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.328507 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:24.328563 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.328643 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.328819 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.332554 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 249) took 1343 ms. Trace:
W20260709 13:18:24.332937 18746 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:18:24.333633 18762 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.333173 18733 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:24.337841 18528 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.317698 18715 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:24.342000 18673 consensus_queue.cc:1243] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:24.406778 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2191) took 2272 ms. Trace:
I20260709 13:18:24.485539 16588 rpcz_store.cc:276] 0709 13:18:22.134760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.392546 (+2257786us) service_pool.cc:224] Handling call
0709 13:18:24.406027 (+ 13481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.406255 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.406291 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.406348 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:24.406394 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.406528 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.406699 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.415601 16708 leader_election.cc:400] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:24.323132 18592 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:24.498835 16745 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:24.505245 16666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 961) took 1796 ms (client timeout 1833 ms). Trace:
W20260709 13:18:24.505725 16666 rpcz_store.cc:269] 0709 13:18:22.708715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.709399 (+   684us) service_pool.cc:224] Handling call
0709 13:18:24.505023 (+1795624us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b54001f7554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.505170 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96301952}
I20260709 13:18:24.505734 18528 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:24.507555 16593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1662) took 3532 ms (client timeout 3035 ms). Trace:
W20260709 13:18:24.508001 16593 rpcz_store.cc:269] 0709 13:18:20.975094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.000473 (+ 25379us) service_pool.cc:224] Handling call
0709 13:18:24.507305 (+3506832us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b5400411254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.507473 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109714560}
I20260709 13:18:24.509307 16593 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.510073 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2193) took 1973 ms. Trace:
I20260709 13:18:24.510399 16593 rpcz_store.cc:276] 0709 13:18:22.536912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.508778 (+1971866us) service_pool.cc:224] Handling call
0709 13:18:24.509091 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.509773 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.510007 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.405332 18755 raft_consensus.cc:2764] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 13:18:24.321444 18740 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:24.393565 18666 raft_consensus.cc:2764] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:24.488157 16708 leader_election.cc:403] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:24.409027 18235 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.513209 16593 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.394377 17411 rpcz_store.cc:276] 0709 13:18:22.989125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.251317 (+1262192us) service_pool.cc:224] Handling call
0709 13:18:24.294798 (+ 43481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.330120 (+ 35322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.332473 (+  2353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.518329 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2194) took 1975 ms. Trace:
I20260709 13:18:24.518839 16593 rpcz_store.cc:276] 0709 13:18:22.543216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.512433 (+1969217us) service_pool.cc:224] Handling call
0709 13:18:24.512832 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.517977 (+  5145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.518253 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:24.514362 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 249426us
I20260709 13:18:24.521665 16588 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:24.522279 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2192) took 1985 ms. Trace:
I20260709 13:18:24.522598 16588 rpcz_store.cc:276] 0709 13:18:22.536627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.487471 (+1950844us) service_pool.cc:224] Handling call
0709 13:18:24.521503 (+ 34032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.522066 (+   563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.522230 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:24.525290 18756 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.228s	user 0.000s	sys 0.000s
I20260709 13:18:24.525666 18611 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.526412 18235 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:24.527292 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 111860us
I20260709 13:18:24.527837 18611 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:24.528165 18592 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:24.528535 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.528785 18592 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 2. Current term is 3.
I20260709 13:18:24.529392 18592 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.529603 18719 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:24.530160 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:24.532363 18694 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 2.035s	user 0.007s	sys 0.000s
W20260709 13:18:24.533017 16671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 968) took 1821 ms (client timeout 1665 ms). Trace:
W20260709 13:18:24.533422 16671 rpcz_store.cc:269] 0709 13:18:22.711636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.747258 (+ 35622us) service_pool.cc:224] Handling call
0709 13:18:24.532835 (+1785577us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540018f954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.532938 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603196672}
W20260709 13:18:24.537048 18765 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:18:24.537117 18616 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:24.537617 18616 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:24.517572 18760 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 13:18:24.539271 16740 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 851) took 1995 ms (client timeout 1870 ms). Trace:
W20260709 13:18:24.539740 16740 rpcz_store.cc:269] 0709 13:18:22.544035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.552116 (+  8081us) service_pool.cc:224] Handling call
0709 13:18:24.539107 (+1986991us) spinlock_profiling.cc:243] Waited 9.07 ms on lock 0x7b54002b9dd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.539207 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20857216}
I20260709 13:18:24.405040 18742 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.542100 16593 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:24.542687 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2195) took 1995 ms. Trace:
I20260709 13:18:24.543015 16593 rpcz_store.cc:276] 0709 13:18:22.546955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.519864 (+1972909us) service_pool.cc:224] Handling call
0709 13:18:24.520266 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.542460 (+ 22194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.542637 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.543843 18556 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:24.544891 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.545750 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:24.548851 16587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1686) took 3101 ms (client timeout 1506 ms). Trace:
I20260709 13:18:24.549880 16591 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:18:24.549276 16587 rpcz_store.cc:269] 0709 13:18:21.447815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.705066 (+257251us) service_pool.cc:224] Handling call
0709 13:18:24.548661 (+2843595us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54003b74d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.548786 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601785600}
I20260709 13:18:24.552692 16587 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.552807 16708 leader_election.cc:400] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4.
I20260709 13:18:24.553669 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.556079 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.557478 16587 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:24.559767 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.565689 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 117) took 2730 ms. Trace:
I20260709 13:18:24.567054 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.567816 16666 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 2.
I20260709 13:18:24.569501 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:24.570065 16737 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.570835 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:24.571446 17415 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:24.571614 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:24.571813 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:24.549471 18648 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.572551 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.579916 16670 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:24.551731 18745 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.582405 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:24.566215 16590 rpcz_store.cc:276] 0709 13:18:21.835826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.303652 (+2467826us) service_pool.cc:224] Handling call
0709 13:18:24.304339 (+   687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.304674 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:24.318466 (+ 13792us) log.cc:844] Serialized 14 byte log entry
0709 13:18:24.565282 (+246816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:24.565331 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:24.565379 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.565471 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:24.565632 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25020,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":97,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":77,"spinlock_wait_cycles":42880,"thread_start_us":246209,"threads_started":1,"wal-append.queue_time_us":246488}
I20260709 13:18:24.582590 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:24.583469 16739 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:24.583583 16742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 186) took 2185 ms (client timeout 1774 ms). Trace:
W20260709 13:18:24.584054 16742 rpcz_store.cc:269] 0709 13:18:22.398278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.404824 (+  6546us) service_pool.cc:224] Handling call
0709 13:18:24.583423 (+2178599us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54003d4354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.583521 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307082496}
I20260709 13:18:24.585054 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:24.585148 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.585585 16742 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:24.587407 16665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 963) took 1877 ms (client timeout 1844 ms). Trace:
I20260709 13:18:24.577452 18755 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:24.579118 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.580000 16587 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.580585 17415 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:24.593519 16669 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:24.580638 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:24.575739 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.574177 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:24.596509 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.596575 16740 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:24.596969 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:24.587800 16665 rpcz_store.cc:269] 0709 13:18:22.709869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.709996 (+   127us) service_pool.cc:224] Handling call
0709 13:18:24.587201 (+1877205us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b540009db54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:24.587336 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120141568}
I20260709 13:18:24.538460 18616 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.598367 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:24.610502 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:24.611136 16671 raft_consensus.cc:606] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:24.611461 16671 raft_consensus.cc:614] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:24.566926 18762 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:24.619530 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:24.610313 16741 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:24.620921 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:24.621465 16741 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:24.622269 16737 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:24.622776 16736 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:24.615182 16742 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:24.619467 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:24.648257 16744 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.619626 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:24.624073 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:24.624961 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:24.628079 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.650308 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:24.590601 18648 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:24.592903 18731 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.612537 18670 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:18:24.624191 18755 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.644660 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.632278 18616 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:24.667866 17416 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:18:24.668551 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 373) took 2670 ms. Trace:
I20260709 13:18:24.630541 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:24.646741 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53595us
W20260709 13:18:24.647166 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.651320 16739 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:24.652905 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:24.653743 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:24.643958 18698 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:24.656652 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.657351 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.657779 16665 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:24.679168 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:24.660727 16744 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:24.667315 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:24.670953 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:24.668884 17416 rpcz_store.cc:276] 0709 13:18:21.998239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.667001 (+2668762us) service_pool.cc:224] Handling call
0709 13:18:24.667447 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.668288 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.668494 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:24.683709 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.683744 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.662106 18544 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:24.685075 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.685451 18544 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:24.656433 18648 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.690405 16669 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:24.691722 18581 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.692332 18581 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.692615 16669 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:24.692881 18581 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.695295 18530 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:24.676963 18638 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:24.697849 18638 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:24.698472 18638 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:24.703675 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.169s (SENT)
I20260709 13:18:24.704262 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.705896 16739 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.709452 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:24.713313 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.685647 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.675840 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:24.718655 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:24.656167 18647 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:24.722651 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.677479 18666 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:24.680915 18745 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:18:24.686496 18768 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:24.726447 18764 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:24.740579 16738 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:39096
I20260709 13:18:24.685895 18544 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:24.685091 17413 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:24.725898 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 953) took 2672 ms. Trace:
W20260709 13:18:24.688642 18778 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:18:24.727077 18738 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:24.728132 16671 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:24.760267 16742 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:24.729532 18742 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:24.731107 18638 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:24.734357 18715 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.686971 18698 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.735375 17416 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.903856 16744 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:24.904381 16744 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:24.905412 16744 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.760187 18738 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:24.741286 16738 raft_consensus.cc:606] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.689240 16664 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:24.912514 16738 raft_consensus.cc:614] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:24.741875 16663 rpcz_store.cc:276] 0709 13:18:22.053410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.055312 (+  1902us) service_pool.cc:224] Handling call
0709 13:18:22.070354 (+ 15042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.870910 (+1800556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.725587 (+854677us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540009d154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:24.725793 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4109460736}
I20260709 13:18:24.662041 18731 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:24.739281 18780 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:24.735314 18670 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:24.895287 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 131517us
I20260709 13:18:24.907662 16744 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:24.739374 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63657.9us
W20260709 13:18:24.953809 16708 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.506s (SENT)
W20260709 13:18:24.978157 16708 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.376s (SENT)
I20260709 13:18:24.896975 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 424) took 2497 ms. Trace:
I20260709 13:18:25.034729 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:24.734442 18648 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.050717 16741 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:25.051586 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:25.035203 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 99980.6us
I20260709 13:18:24.920421 18738 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:25.074958 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.075546 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.037371 18447 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.099615 18447 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.102063 18447 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:25.105180 18784 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:18:25.105366 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.109040 18698 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:25.043229 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:24.896633 18742 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.037235 18649 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:18:25.037236 18528 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.015053 18670 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:25.079797 16669 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:25.124835 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:25.125588 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 147491us
I20260709 13:18:25.125880 16738 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
W20260709 13:18:25.014575 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.419s (SENT)
W20260709 13:18:25.090986 16589 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.021s	sys 0.022s
W20260709 13:18:25.127737 16589 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.021s	sys 0.022s
I20260709 13:18:25.127645 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:25.128850 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 118) took 3280 ms. Trace:
W20260709 13:18:25.129099 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 114407us
W20260709 13:18:25.131072 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.035495 18780 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:25.121749 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1674) took 2160 ms. Trace:
I20260709 13:18:25.129316 16589 rpcz_store.cc:276] 0709 13:18:21.848798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.304090 (+2455292us) service_pool.cc:224] Handling call
0709 13:18:24.304459 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.304713 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:24.556220 (+251507us) log.cc:844] Serialized 14 byte log entry
0709 13:18:25.128404 (+572184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:25.128476 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:25.128538 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.128626 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.128778 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15917,"prepare.queue_time_us":27329,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"thread_start_us":798101,"threads_started":2}
I20260709 13:18:25.132009 16662 rpcz_store.cc:276] 0709 13:18:22.961288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.996068 (+ 34780us) service_pool.cc:224] Handling call
0709 13:18:22.996410 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.182087 (+1185677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.121506 (+939419us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540001a254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:25.121685 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2724938496}
I20260709 13:18:25.133430 18528 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.135972 18528 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:25.143653 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:25.143570 16738 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:18:25.154129 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.566s (SENT)
I20260709 13:18:25.154747 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:25.156344 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.144480 16662 raft_consensus.cc:606] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:25.167650 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.173174 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:18:25.132254 18670 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.141932 18745 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:25.145299 17414 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:25.046820 17413 rpcz_store.cc:276] 0709 13:18:22.399692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.684276 (+2284584us) service_pool.cc:224] Handling call
0709 13:18:24.684598 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.748389 (+ 63791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:24.894833 (+146444us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:25.162951 18728 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:25.130928 18581 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:25.212152 16559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.009s	sys 0.038s
W20260709 13:18:25.213331 16559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.009s	sys 0.038s
W20260709 13:18:25.213851 16559 connection.cc:256] RPC call timeout handler was delayed by -1.3039374440002121s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
W20260709 13:18:25.214217 16559 connection.cc:256] RPC call timeout handler was delayed by -1.2040374440000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
I20260709 13:18:25.214301 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:25.214613 16559 connection.cc:256] RPC call timeout handler was delayed by -1.152737444000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
W20260709 13:18:25.215042 16559 connection.cc:256] RPC call timeout handler was delayed by -1.1428374440001789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
I20260709 13:18:25.215071 16738 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
W20260709 13:18:25.215508 16559 connection.cc:256] RPC call timeout handler was delayed by -1.1194374440001411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
I20260709 13:18:25.220384 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2126) took 1377 ms. Trace:
I20260709 13:18:25.201301 16744 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:18:25.221096 16744 pending_rounds.cc:85] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 2
I20260709 13:18:25.221379 16744 pending_rounds.cc:107] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:18:25.221618 16744 raft_consensus.cc:2889] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.220844 16743 rpcz_store.cc:276] 0709 13:18:23.843077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.945130 (+102053us) service_pool.cc:224] Handling call
0709 13:18:23.954406 (+  9276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.955246 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.220167 (+1264921us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b54003d3950. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:25.220318 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20607744}
W20260709 13:18:25.259294 16737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.021s	sys 0.086s
W20260709 13:18:25.259724 16737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.021s	sys 0.086s
I20260709 13:18:25.260038 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.262785 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.263933 16593 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:25.340513 16589 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:25.341712 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.343281 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.344455 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:25.333909 17410 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.037s	sys 0.047s
W20260709 13:18:25.349881 17410 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.038s	sys 0.047s
W20260709 13:18:25.350096 18781 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:18:25.351542 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2201) took 1382 ms. Trace:
I20260709 13:18:25.351980 16588 rpcz_store.cc:276] 0709 13:18:23.968823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.523331 (+554508us) service_pool.cc:224] Handling call
0709 13:18:24.547058 (+ 23727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:24.547393 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:24.547421 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:24.552351 (+  4930us) log.cc:844] Serialized 26 byte log entry
0709 13:18:25.042109 (+489758us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:25.042155 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.042235 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.351302 (+309067us) raft_consensus.cc:1685] finished
0709 13:18:25.351326 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.351482 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":454,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4286,"thread_start_us":449384,"threads_started":1,"wal-append.queue_time_us":595420}
I20260709 13:18:25.353856 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:25.355453 18782 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:18:25.356251 18780 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.361713 18742 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:25.363817 18581 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:25.364915 18647 consensus_meta.cc:287] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.000s
I20260709 13:18:25.365370 18647 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:25.367205 18731 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:25.339622 18728 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.371338 18728 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:25.373214 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 59178.2us
I20260709 13:18:25.348062 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.377103 16589 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:25.378402 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:25.382299 18559 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.383193 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:25.383766 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.384706 16589 raft_consensus.cc:2410] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:25.385464 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.382921 18559 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.392158 16590 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:25.393246 16594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1694) took 3873 ms (client timeout 2376 ms). Trace:
I20260709 13:18:25.394047 18559 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:25.393733 16594 rpcz_store.cc:269] 0709 13:18:21.519368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.036125 (+516757us) service_pool.cc:224] Handling call
0709 13:18:25.391841 (+3355716us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400339fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:25.393165 (+  1324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643205376}
I20260709 13:18:25.396040 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.397352 16587 raft_consensus.cc:2410] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:25.397420 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:25.395318 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.387452 16589 raft_consensus.cc:2376] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:25.340181 18600 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.394851 18648 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:25.400581 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.404078 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.407372 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.408196 16589 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:25.409473 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.410378 16589 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 2: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:25.411293 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.407125 16587 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:25.412972 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.414943 16589 raft_consensus.cc:2410] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:25.415799 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:25.404441 18559 raft_consensus.cc:2764] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:25.421432 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.422418 16587 raft_consensus.cc:2410] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:25.424088 16589 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:25.405498 18600 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:25.426270 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 299764us
W20260709 13:18:25.427349 16668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 941) took 5434 ms (client timeout 1840 ms). Trace:
W20260709 13:18:25.427832 16668 rpcz_store.cc:269] 0709 13:18:19.990854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.003326 (+ 12472us) service_pool.cc:224] Handling call
0709 13:18:25.424571 (+5421245us) spinlock_profiling.cc:243] Waited 4.03 s on lock 0x7b540000ffd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:25.424695 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9260757120}
I20260709 13:18:25.398897 18785 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:18:25.429987 16594 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:25.430644 18544 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.000s
I20260709 13:18:25.431032 18600 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:25.431227 18544 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:25.432525 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.432868 16670 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:25.433452 18648 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:25.434053 18648 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.440850 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:25.451148 16559 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:25.453745 16559 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:25.462934 18745 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:25.478302 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 198) took 1302 ms. Trace:
I20260709 13:18:25.478849 16745 rpcz_store.cc:276] 0709 13:18:24.175454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.175607 (+   153us) service_pool.cc:224] Handling call
0709 13:18:24.175949 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.499548 (+323599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.478014 (+978466us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400307354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:25.478235 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740927744}
W20260709 13:18:25.483366 18692 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 2.987s	user 0.004s	sys 0.034s
I20260709 13:18:25.483839 18556 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:25.484274 18556 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:25.485117 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 240) took 3584 ms. Trace:
I20260709 13:18:25.485745 17415 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Advancing to term 4
I20260709 13:18:25.485494 18556 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.488006 18648 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:25.438469 18600 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:25.495244 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.485538 17420 rpcz_store.cc:276] 0709 13:18:21.900545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:21.900669 (+   124us) service_pool.cc:224] Handling call
0709 13:18:21.920646 (+ 19977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:21.935200 (+ 14554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:21.935233 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:21.935758 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 13:18:22.414407 (+478649us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:22.490726 (+ 76319us) log.cc:844] Serialized 14 byte log entry
0709 13:18:22.491002 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:22.491072 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:25.428783 (+2937711us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:25.484747 (+ 55964us) raft_consensus.cc:1685] finished
0709 13:18:25.484868 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:25.485053 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70949,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":1208,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":27326,"spinlock_wait_cycles":43841408,"thread_start_us":437426,"threads_started":1,"wal-append.queue_time_us":560133}
W20260709 13:18:25.496642 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 96816.7us
W20260709 13:18:25.499094 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.502022 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:25.492089 18785 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:25.502698 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64624.9us
I20260709 13:18:25.498283 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:25.516841 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:25.493445 18600 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:25.494014 18740 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:25.514827 18556 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:25.526927 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.503734 18648 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:25.517324 18728 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:25.511786 17415 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:18:25.543318 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.549535 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.547s (SENT)
I20260709 13:18:25.522301 18681 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:25.550076 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:25.531773 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:25.560027 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:25.557578 18795 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:25.561261 18795 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.561852 18795 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:25.563730 18670 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.557476 18681 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:25.566011 18681 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.567418 18681 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.550575 18544 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:25.571273 18803 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:25.551007 18648 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.576069 18670 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:25.581395 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 387297us
I20260709 13:18:25.548532 18646 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.591006 18711 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:18:25.591706 18711 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:25.592173 18711 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:25.595036 18600 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.596797 18343 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.597550 18343 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:25.600081 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 101872us
W20260709 13:18:25.599812 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 146177us
I20260709 13:18:25.602123 16559 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:25.610373 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:25.613994 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:25.620396 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70899.8us
I20260709 13:18:25.624796 18343 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:25.619941 18792 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:18:25.599753 18648 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:25.618705 18434 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:25.629207 16708 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.762s (SENT)
I20260709 13:18:25.619681 18646 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:25.632860 16708 leader_election.cc:336] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.005s (SENT)
I20260709 13:18:25.636483 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:25.618320 18546 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:18:25.619897 18738 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.699836 16589 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:25.701327 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:25.702982 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 67066.5us
I20260709 13:18:25.704977 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:25.632954 18648 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:25.714671 16588 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:25.715616 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.718732 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.724480 16587 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.725929 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:25.647047 18796 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.731950 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:25.730214 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.716265 18647 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.736202 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.745850 16589 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:25.633946 18434 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.733359 18343 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:25.759284 18343 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:25.760295 18343 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.761080 18365 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:25.760844 18647 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.761682 18365 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:25.758606 16589 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:25.749397 16587 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:18:25.750924 17412 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 244) took 3210 ms (client timeout 2048 ms). Trace:
I20260709 13:18:25.764014 18647 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:25.764489 17412 rpcz_store.cc:269] 0709 13:18:22.540548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.620404 (+ 79856us) service_pool.cc:224] Handling call
0709 13:18:25.750603 (+3130199us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b5400465354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:25.750810 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1989857536}
I20260709 13:18:25.765406 18581 raft_consensus.cc:993] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:18:25.765954 18581 raft_consensus.cc:1081] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:18:25.762681 18365 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.766837 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.736867 18803 consensus_queue.cc:1243] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:18:25.733958 18785 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:25.768586 18785 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.771406 18802 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:25.773499 18646 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.775949 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:25.776674 18434 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:25.776721 16589 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:25.774300 18722 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 2.015s	user 0.010s	sys 0.015s
W20260709 13:18:25.778239 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:25.779306 16668 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:25.787117 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:25.787849 16745 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:25.748294 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:23.905646 16596 rpcz_store.cc:276] 0709 13:18:21.776059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.658119 (+1882060us) service_pool.cc:224] Handling call
0709 13:18:23.670056 (+ 11937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:23.670242 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:23.853252 (+183010us) log.cc:844] Serialized 14 byte log entry
0709 13:18:23.901610 (+ 48358us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:23.901668 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:23.901714 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:23.901786 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:23.901924 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":533,"prepare.queue_time_us":247222,"raft.queue_time_us":2384,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2565,"thread_start_us":191293,"threads_started":2}
I20260709 13:18:25.780675 16587 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:25.790169 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:25.819244 16592 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:18:25.821094 16592 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:25.821578 16592 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:25.823632 16596 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:25.823339 16592 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:25.823724 16587 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:25.826210 16592 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:25.827150 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:25.785473 18618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.821s	user 0.032s	sys 0.036s
I20260709 13:18:25.735476 18605 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 13:18:25.855330 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 175) took 1150 ms. Trace:
I20260709 13:18:25.855942 16670 rpcz_store.cc:276] 0709 13:18:24.705004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.722343 (+ 17339us) service_pool.cc:224] Handling call
0709 13:18:25.855045 (+1132702us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b5400002154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:25.855242 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1630726400}
I20260709 13:18:25.859205 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2213) took 1860 ms. Trace:
I20260709 13:18:25.859632 16590 rpcz_store.cc:276] 0709 13:18:23.998475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.583191 (+584716us) service_pool.cc:224] Handling call
0709 13:18:25.391967 (+808776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.392666 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.858949 (+466283us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b5400339fd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:25.859128 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859342720}
I20260709 13:18:25.860901 16592 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:18:25.861310 16592 pending_rounds.cc:85] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 3
I20260709 13:18:25.861524 16592 pending_rounds.cc:107] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:18:25.861743 16592 raft_consensus.cc:2889] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:25.822783 18719 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:25.832181 18618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.868s	user 0.032s	sys 0.036s
W20260709 13:18:25.877730 16710 connection.cc:256] RPC call timeout handler was delayed by -1.2881168739997975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
I20260709 13:18:25.770756 18711 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.826535 18745 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:18:25.889739 16710 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:25.825001 18647 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:25.873075 18680 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.834710 18434 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:18:25.872534 18670 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:25.941037 18670 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.943395 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:25.922410 18719 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:18:25.944123 17416 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:25.945837 18719 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:25.948863 18235 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:25.895488 18762 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:25.938937 18711 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:25.939359 18647 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:25.939424 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:25.843884 18447 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 13:18:25.936529 16589 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:25.900936 18648 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:25.959367 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 311287us
W20260709 13:18:25.962199 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 257360us
I20260709 13:18:25.964689 18803 raft_consensus.cc:2764] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:18:25.881893 18618 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:25.965461 16670 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:25.979924 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:25.959488 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:25.962561 17409 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:25.963104 18647 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:25.968247 17413 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:26.001883 16708 leader_election.cc:336] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 3.057s (SENT)
I20260709 13:18:25.944150 18680 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.003680 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.003696 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:25.959421 18795 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:25.968312 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:26.002038 16745 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260709 13:18:26.007699 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.027536 16737 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:26.028669 16561 connection.cc:256] RPC call timeout handler was delayed by -1.9607134100001531s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1665s for a response
I20260709 13:18:25.994612 18670 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.025409 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:25.961164 18762 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:26.029122 16561 connection.cc:256] RPC call timeout handler was delayed by -1.79961341000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
I20260709 13:18:25.991006 18711 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:18:25.998386 18640 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:18:25.967921 18648 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:26.036315 16561 connection.cc:256] RPC call timeout handler was delayed by -1.7762134100000821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
I20260709 13:18:25.986454 17416 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:26.037305 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:26.053917 16561 connection.cc:256] RPC call timeout handler was delayed by -1.5584134100001847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 13:18:26.054366 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4999134100000902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
W20260709 13:18:26.054724 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4206208179998612s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1796s for a response
W20260709 13:18:26.055068 16561 connection.cc:256] RPC call timeout handler was delayed by -1.187520817999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
W20260709 13:18:26.055393 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1110208180000427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 13:18:26.001240 18799 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.056524 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:26.057195 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:26.057560 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:26.057865 16590 raft_consensus.cc:2376] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5. Re-sending same reply.
I20260709 13:18:26.058328 16592 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.059034 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:26.062347 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:26.041504 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 190964us
I20260709 13:18:26.001938 18738 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.053416 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:26.037606 16745 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.001063 18600 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:18:26.067250 16737 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.029542 18807 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:18:26.075837 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 186192us
I20260709 13:18:26.076855 16742 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.050674 18805 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.081249 18805 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.079488 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:26.034833 17414 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:26.024049 18605 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:18:26.039382 18547 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:26.068041 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:26.093822 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.072677 18711 consensus_queue.cc:1243] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:26.095525 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:26.097546 16743 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.100071 16745 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:26.098383 16737 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:26.105809 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.108129 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:26.102772 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 230696us
W20260709 13:18:26.120116 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.109117 16742 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:26.108979 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 89937.4us
I20260709 13:18:26.087319 18365 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:26.125993 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58024.1us
I20260709 13:18:26.128576 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.111123 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:26.097612 18807 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:26.130666 16592 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:26.137851 16595 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:26.140352 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:26.067548 18618 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.145036 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.145550 18807 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.145036 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:26.147336 16737 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:26.104923 18812 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:26.150516 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.131273 18605 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:26.153363 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 146051us
I20260709 13:18:26.153472 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.149813 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.161062 16745 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:26.131788 18365 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:26.142756 18800 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.190s	user 0.002s	sys 0.000s
I20260709 13:18:26.156780 16662 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:26.166340 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.166998 16737 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:26.172510 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 88054.3us
I20260709 13:18:26.172652 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.148752 18600 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:26.148342 18795 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:26.175107 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:26.150375 18785 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:26.175150 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.174376 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:26.171068 18813 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:18:26.186146 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.185650 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.189380 16737 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:26.189500 16745 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
W20260709 13:18:26.186045 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 148808us
I20260709 13:18:26.187764 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.190949 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.190511 18240 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
W20260709 13:18:26.217557 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.190251 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:26.204613 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.203841 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:26.213805 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:26.180616 18542 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.203899 18764 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.190212 18785 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.242086 16742 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:26.244004 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:26.245273 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:26.242163 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58455.7us
I20260709 13:18:26.296986 17416 raft_consensus.cc:2410] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:26.192377 18646 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.204559 18738 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:26.266769 16745 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:26.250279 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55155.7us
I20260709 13:18:26.156419 18812 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.174274 18807 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:18:26.312601 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:26.297472 16634 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:26.295249 16742 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:26.301644 18826 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 13:18:26.303021 18815 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:18:26.308750 18812 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:26.298265 18785 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.236531 18365 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.249454 18542 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.311131 18600 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.183153 18605 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:18:26.313355 17416 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:26.314436 16742 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.244668 18670 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.312085 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:26.320303 18822 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:18:26.293460 18648 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:26.323977 18820 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:26.297849 18556 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:18:26.425738 18556 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:26.297703 18646 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:26.244786 18814 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:18:26.192858 18805 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.215975 18658 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:26.336251 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:26.241712 18680 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.242386 18762 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:26.250761 18711 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:26.345734 16742 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.243393 18823 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:26.338091 18811 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.245s	user 0.001s	sys 0.001s
I20260709 13:18:26.251961 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:26.344115 18542 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:26.500873 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.347790 18670 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:26.347378 18819 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:18:26.338544 18240 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 13:18:26.350703 18755 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:18:26.351125 18807 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.511204 18622 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:18:26.511698 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 175602us
I20260709 13:18:26.513353 18648 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:18:26.512660 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:26.351454 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1681) took 1312 ms. Trace:
I20260709 13:18:26.351601 17409 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:26.516780 16667 rpcz_store.cc:276] 0709 13:18:25.038755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.042158 (+  3403us) service_pool.cc:224] Handling call
0709 13:18:25.042646 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:25.057958 (+ 15312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.058002 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:25.476241 (+418239us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b2c00010da8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:18:25.476856 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 13:18:25.958568 (+481712us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:25.958618 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:25.958721 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:26.347497 (+388776us) raft_consensus.cc:1685] finished
0709 13:18:26.347516 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:26.351383 (+  3867us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14702,"raft.queue_time_us":16400,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":146,"spinlock_wait_cycles":59899648,"thread_start_us":481251,"threads_started":1,"wal-append.queue_time_us":612103}
I20260709 13:18:26.518376 18795 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.520048 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:26.520807 18823 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:26.522667 18823 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:26.346745 18827 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:26.524282 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.523820 18827 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.528198 18806 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:26.528827 18806 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:26.529531 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.242949 18809 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
W20260709 13:18:26.355890 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44074.7us
I20260709 13:18:26.356531 18835 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:26.357906 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 169413us
I20260709 13:18:26.358232 18834 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:26.364208 18832 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:26.366004 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 987) took 1762 ms. Trace:
I20260709 13:18:26.546236 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:26.350760 17413 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:26.459321 17416 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:26.299420 18764 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.346755 18818 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.135s
I20260709 13:18:26.477418 18646 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.501770 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:26.481010 18658 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.489151 18711 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:26.523226 18823 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:26.530285 18820 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:26.534221 18762 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.560549 18807 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.598831 18614 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.546241 16669 rpcz_store.cc:276] 0709 13:18:24.603909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.693628 (+ 89719us) service_pool.cc:224] Handling call
0709 13:18:24.693987 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.124590 (+430603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.365669 (+1241079us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54001b08d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:26.365925 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":886314240}
I20260709 13:18:26.506603 18670 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.629612 18614 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.630352 16493 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 138) took 11051 ms. Trace:
I20260709 13:18:26.526737 18827 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:26.622077 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1711) took 1284 ms. Trace:
I20260709 13:18:26.631762 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:26.580142 18810 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:26.632259 18810 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:26.380204 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.633250 18810 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.634246 18810 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:26.634378 18715 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:26.587397 17412 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:26.579350 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1688) took 5094 ms. Trace:
I20260709 13:18:26.548377 18830 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:26.644342 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.655455 16738 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:26.655180 16744 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:26.631294 16594 rpcz_store.cc:276] 0709 13:18:25.380967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.395032 (+ 14065us) service_pool.cc:224] Handling call
0709 13:18:26.621820 (+1226788us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5400051754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:26.622004 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70759168}
I20260709 13:18:26.575700 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:26.659384 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 415544us
I20260709 13:18:26.636319 16591 rpcz_store.cc:276] 0709 13:18:21.484960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:22.259425 (+774465us) service_pool.cc:224] Handling call
0709 13:18:24.549572 (+2290147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:24.550319 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.579002 (+2028683us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54003b74d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:26.579224 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5265816832}
I20260709 13:18:26.658738 16745 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:26.663892 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.613756 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2150) took 2038 ms. Trace:
I20260709 13:18:26.637027 16492 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:26.664583 16741 rpcz_store.cc:276] 0709 13:18:24.575400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.621991 (+ 46591us) service_pool.cc:224] Handling call
0709 13:18:24.622226 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.066714 (+444488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.613479 (+1546765us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54003d4854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:26.613688 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":984861568}
I20260709 13:18:26.541755 16742 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:18:26.666641 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:26.617316 18821 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.433s	user 0.003s	sys 0.000s
I20260709 13:18:26.574939 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 385) took 1814 ms. Trace:
I20260709 13:18:26.671578 17415 rpcz_store.cc:276] 0709 13:18:24.760022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.761415 (+  1393us) service_pool.cc:224] Handling call
0709 13:18:25.485613 (+724198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:25.549605 (+ 63992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.574566 (+1024961us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400681350. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:26.574818 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357707008}
W20260709 13:18:26.672137 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 170468us
W20260709 13:18:26.603799 18796 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.865s	user 0.009s	sys 0.006s
I20260709 13:18:26.666909 16742 pending_rounds.cc:85] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 5
I20260709 13:18:26.665588 16492 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:26.672808 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:26.674142 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:26.588012 18665 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:18:26.670473 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:26.740552 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:26.742475 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:26.562028 18658 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:26.744011 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 73581.6us
I20260709 13:18:26.672278 18830 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.665997 18711 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:26.769318 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:26.756726 17374 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.151s (SENT)
I20260709 13:18:26.760241 16742 pending_rounds.cc:107] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:18:26.673085 18807 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.772528 16742 raft_consensus.cc:2889] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:26.773576 18238 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:26.773927 18780 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:18:26.619122 18764 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.669986 18808 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:26.776427 18808 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.778939 18808 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.670156 18740 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.769845 18600 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.779712 18740 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.781538 18805 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:26.782052 18740 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:26.744463 18796 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.782994 18699 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:26.783313 16588 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
W20260709 13:18:26.760313 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:26.761067 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 417763us
I20260709 13:18:26.672432 18731 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:26.746559 18709 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:26.784299 16634 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:26.765304 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:26.785012 18709 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:26.785131 18830 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:26.786023 18709 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.737425 18605 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.795573 18807 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.798456 18605 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.800601 18807 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:26.801529 18605 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.802296 18805 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:26.803148 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 130391us
I20260709 13:18:26.804282 18843 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.804760 18843 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:26.805162 18843 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:26.806200 18556 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.806800 18556 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:26.746091 18799 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.672s	user 0.006s	sys 0.006s
I20260709 13:18:26.807879 18799 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.806921 18670 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.675525 18640 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:26.765434 16665 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
W20260709 13:18:26.767674 18647 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.000s
I20260709 13:18:26.740011 18826 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.809928 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:26.787550 18817 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.658s	user 0.002s	sys 0.000s
I20260709 13:18:26.813112 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:26.813928 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:26.846544 16594 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:26.814694 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:26.848140 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:26.848557 16587 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:26.834758 16669 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:26.835698 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:26.851768 16667 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:26.836624 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.839672 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:26.810165 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.840797 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.784256 18731 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:26.844429 16665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 982) took 2224 ms (client timeout 2049 ms). Trace:
W20260709 13:18:26.890048 16665 rpcz_store.cc:269] 0709 13:18:24.602669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.680162 (+ 77493us) service_pool.cc:224] Handling call
0709 13:18:26.575562 (+1895400us) spinlock_profiling.cc:243] Waited 3.92 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabff8c 00007f327cabfef6 0000000000375939 00007f3284b331e0 00007f3284bfc0c4 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:26.827475 (+251913us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9031936}
I20260709 13:18:26.671010 18762 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.826225 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:26.861693 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.782490 18646 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:26.901373 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.671597 18542 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.853044 16668 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:26.911408 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:26.812384 16592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:18:26.912456 16592 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:26.912937 16592 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:26.883364 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.885834 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:26.915868 16739 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:26.916661 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.917721 16669 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:26.604638 18806 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.891620 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:26.931365 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:26.933467 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:26.767812 18666 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
W20260709 13:18:26.895089 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 230330us
W20260709 13:18:26.896289 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48271.4us
I20260709 13:18:26.935999 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.843941 18831 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.845888 16591 raft_consensus.cc:2376] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5. Re-sending same reply.
I20260709 13:18:26.630748 16493 rpcz_store.cc:276] 0709 13:18:15.579301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:15.579760 (+   459us) service_pool.cc:224] Handling call
0709 13:18:26.630289 (+11050529us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:18.028599 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:18.046666 (+ 18067us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:18.046739 (+    73us) write_op.cc:435] Acquired schema lock
0709 13:18:18.046782 (+    43us) tablet.cc:662] Decoding operations
0709 13:18:18.048473 (+  1691us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:18.048625 (+   152us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:18.048669 (+    44us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:18:18.050249 (+  1580us) tablet.cc:701] Row locks acquired
0709 13:18:18.050317 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:18:18.050580 (+   263us) write_op.cc:270] Start()
0709 13:18:18.050778 (+   198us) write_op.cc:276] Timestamp: P: 1783603098050496 usec, L: 0
0709 13:18:18.050836 (+    58us) op_driver.cc:348] REPLICATION: starting
0709 13:18:18.261833 (+210997us) log.cc:844] Serialized 4362 byte log entry
0709 13:18:18.926913 (+665080us) op_driver.cc:464] REPLICATION: finished
0709 13:18:19.141375 (+214462us) write_op.cc:301] APPLY: starting
0709 13:18:19.141484 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:19.141743 (+   259us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:19.178731 (+ 36988us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:19.363309 (+184578us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:19.363708 (+   399us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:18:19.363796 (+    88us) write_op.cc:312] APPLY: finished
0709 13:18:19.364191 (+   395us) log.cc:844] Serialized 83 byte log entry
0709 13:18:19.364391 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:19.364548 (+   157us) write_op.cc:454] Released schema lock
0709 13:18:19.364619 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:19.365089 (+   470us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:19.365121 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":606,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":208980,"apply.run_cpu_time_us":5407,"apply.run_wall_time_us":223720,"num_ops":11,"prepare.queue_time_us":380242,"prepare.run_cpu_time_us":48017,"prepare.run_wall_time_us":674259,"raft.queue_time_us":141596,"raft.run_cpu_time_us":7730,"raft.run_wall_time_us":157897,"replication_time_us":875811,"thread_start_us":388813,"threads_started":4,"wal-append.queue_time_us":359597,"wal-append.run_cpu_time_us":14077,"wal-append.run_wall_time_us":1680480}]]}
I20260709 13:18:26.902766 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.947415 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:26.948112 16662 raft_consensus.cc:2410] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:18:26.905225 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 79113.5us
I20260709 13:18:26.907073 16665 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:26.768500 16492 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:26.830322 18640 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:26.871102 18805 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:26.937252 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:26.737407 18836 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.886586 18581 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:26.951845 16592 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.955513 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:26.850560 18647 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.902561 18681 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:26.886257 18758 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:26.958989 16667 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:26.980420 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:26.986054 16664 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:26.986423 16667 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 13:18:26.904341 18796 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:18:26.965956 16662 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:26.880319 18839 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:26.863397 18826 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.903570 18799 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:26.977324 18699 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:27.005681 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:26.976541 16589 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:26.926272 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:26.925733 18795 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.880443 18605 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:18:26.926469 18832 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:26.983623 16668 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:26.959002 18640 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.949390 18434 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:26.964866 18807 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:26.970994 18530 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:26.971141 18793 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:26.862619 18820 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.970738 18830 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.002413 16739 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:26.910447 18614 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.005447 18846 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:18:26.911839 18818 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:26.989045 18600 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:27.044976 18818 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:27.045089 18848 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:18:27.045266 18600 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.045420 18818 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.045706 18600 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.036509 18434 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.028954 18807 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:27.049510 18698 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.050079 18698 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.947407 18738 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:18:27.050426 18807 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.971503 18836 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.051512 18738 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:27.051683 18807 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.051900 18738 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:27.052876 18738 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.054206 18738 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:26.977387 18681 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.055790 18681 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.056696 18681 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.058002 18742 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:26.990715 18758 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.042054 18530 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.059429 18530 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.059631 18758 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.976037 18618 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:18:26.977473 18581 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.062119 18581 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.988793 18693 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:18:26.988945 18834 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:18:27.062963 18794 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:27.004916 18699 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.063534 18794 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:27.063653 18699 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.064428 18794 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.065534 18762 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.009368 18597 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:18:27.012279 18622 consensus_meta.cc:287] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.000s
I20260709 13:18:26.958387 18831 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:26.910930 18823 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.008276 18839 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.010609 18740 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:27.040886 18830 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.042688 18651 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:27.010881 18680 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:18:27.053304 18836 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.028998 18795 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.092868 18651 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.058516 18742 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.012126 18731 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.069830 18793 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:27.070655 18785 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
I20260709 13:18:27.077927 18609 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.196452 18609 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:26.950655 16492 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.105453 18647 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:26.967233 16592 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.183414 18803 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:27.156255 16593 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:27.161705 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:27.013262 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.213735 16669 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:27.104887 18646 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.216643 18605 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.217844 18605 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:27.044055 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50937.3us
I20260709 13:18:27.070655 18597 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.236593 18597 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.090193 18711 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.076417 18670 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.238479 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2149) took 2663 ms. Trace:
I20260709 13:18:27.238411 18597 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.240083 18693 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:27.082381 18826 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:27.271121 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.125206 18831 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.075492 18833 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:27.274308 18833 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.275015 18833 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:27.122944 18839 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.128003 18843 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.297760 17410 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.117411 18785 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.098152 18830 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.303112 17413 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.303809 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.106758 18795 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.058943 18801 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.207s
I20260709 13:18:27.101619 18640 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:27.178723 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1680) took 2143 ms. Trace:
I20260709 13:18:27.098192 18600 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:27.190501 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.304814 16741 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:27.201310 16492 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:27.211969 16591 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.220306 16667 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:27.220691 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 131) took 1164 ms. Trace:
I20260709 13:18:27.091624 18740 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.223408 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.223663 18651 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.223910 16590 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:18:27.008488 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 70546.9us
I20260709 13:18:27.315682 18640 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.315919 16492 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.317718 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:27.309641 16671 rpcz_store.cc:276] 0709 13:18:25.038303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.045082 (+  6779us) service_pool.cc:224] Handling call
0709 13:18:25.045530 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:25.053819 (+  8289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:25.053864 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:26.741332 (+1687468us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b2c00010da8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:18:26.741746 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:18:26.949176 (+207430us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:26.949229 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:26.949515 (+   286us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.177694 (+228179us) raft_consensus.cc:1685] finished
0709 13:18:27.177753 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.178366 (+   613us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7802,"raft.queue_time_us":8458,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"spinlock_wait_cycles":53063936,"thread_start_us":169750,"threads_started":1,"wal-append.queue_time_us":221050}
W20260709 13:18:27.224011 16588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2224) took 1542 ms (client timeout 1709 ms). Trace:
I20260709 13:18:27.319245 18830 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.204023 18803 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:27.227207 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 179144us
I20260709 13:18:27.322479 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.324245 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.227962 18646 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.228631 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 135) took 1167 ms. Trace:
W20260709 13:18:27.319036 16588 rpcz_store.cc:269] 0709 13:18:25.681392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.715925 (+ 34533us) service_pool.cc:224] Handling call
0709 13:18:27.223619 (+1507694us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400320354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.223826 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416207616}
I20260709 13:18:27.325392 16595 rpcz_store.cc:276] 0709 13:18:26.061029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.061170 (+   141us) service_pool.cc:224] Handling call
0709 13:18:26.137516 (+ 76346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.138301 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.228197 (+1089896us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b54002373d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.228444 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174771968}
I20260709 13:18:27.325461 18646 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.210561 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.326982 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.327383 16592 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:27.327661 18740 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.328154 18646 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.328581 17411 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.241034 18832 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.241230 18711 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.241369 16596 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:18:27.247538 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.248860 18343 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.328963 18651 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:27.256880 16744 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 13:18:27.260270 18365 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.123800 18793 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.264468 18542 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.265601 16739 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:27.239255 16740 rpcz_store.cc:276] 0709 13:18:24.575166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.597274 (+ 22108us) service_pool.cc:224] Handling call
0709 13:18:24.597563 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:26.534457 (+1936894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:26.534506 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:26.534568 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:26.534630 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.237738 (+703108us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b540007c6d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:27.237830 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.238200 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4453729024}
I20260709 13:18:27.220763 18530 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.340657 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:27.275928 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.342007 18530 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.342216 16742 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:27.333272 18343 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:27.245085 18693 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:27.343670 18786 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 1.881s	user 0.001s	sys 0.013s
I20260709 13:18:27.006270 16587 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:27.344547 18343 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:27.314679 16594 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:18:27.346453 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.348328 16587 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:27.335413 18843 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.349890 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.317922 16492 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.324860 18803 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:27.230893 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.302961 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:27.352522 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.328493 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.060467 18818 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.353592 16588 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
I20260709 13:18:27.354907 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.355772 18818 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.356215 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.328552 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.310956 16589 rpcz_store.cc:276] 0709 13:18:26.056073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.056236 (+   163us) service_pool.cc:224] Handling call
0709 13:18:27.220080 (+1163844us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54000d98d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.220360 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813801856}
I20260709 13:18:27.331874 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.331888 18832 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.335393 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.336043 16669 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:27.336190 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.360795 16738 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:27.361043 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:27.337074 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.334457 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.348285 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.319948 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:27.350893 16587 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:27.314067 18605 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.363471 16595 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:27.363339 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:27.365010 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.335450 18600 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.367508 16592 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:27.370296 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.359202 18343 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.375924 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:27.376243 16667 raft_consensus.cc:2376] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5. Re-sending same reply.
I20260709 13:18:27.376784 17413 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:27.377549 18823 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:27.377820 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 58545.6us
I20260709 13:18:27.378366 16663 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:27.379552 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:27.379649 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:27.381117 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.379607 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.383088 17412 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:27.383817 18818 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:27.384913 16669 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:18:27.385703 18818 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:27.386279 18818 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.386307 18866 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.376878 16589 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:18:27.373839 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90439.5us
I20260709 13:18:27.320088 18830 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.386740 16669 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:27.391975 16741 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.392792 18719 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.393373 18719 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.395906 18785 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.396173 16595 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:18:27.397375 18698 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.397986 16595 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:18:27.398406 16595 pending_rounds.cc:85] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 4
I20260709 13:18:27.398687 16595 pending_rounds.cc:107] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260709 13:18:27.399263 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58929.4us
W20260709 13:18:27.401631 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84220.8us
W20260709 13:18:27.402279 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.399680 16595 raft_consensus.cc:2889] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:27.385732 18803 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.379667 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:27.416496 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:27.417012 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.418424 18794 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:27.418572 18240 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:27.419399 18240 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.421247 17411 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:27.403788 18605 raft_consensus.cc:2764] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
W20260709 13:18:27.425935 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.427785 17410 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:27.410789 18823 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.409497 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.427448 18803 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.376147 16492 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 4, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:27.455628 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:27.459203 17416 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:27.463192 16745 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:27.463802 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 218946us
W20260709 13:18:27.464041 17419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 423) took 5064 ms (client timeout 3049 ms). Trace:
I20260709 13:18:27.464380 18605 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.465305 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:27.464543 17419 rpcz_store.cc:269] 0709 13:18:22.399425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:23.935884 (+1536459us) service_pool.cc:224] Handling call
0709 13:18:27.463588 (+3527704us) spinlock_profiling.cc:243] Waited 2.88 s on lock 0x7b5400484254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.463811 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6624193792}
I20260709 13:18:27.464862 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:27.475265 17409 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:18:27.475550 17415 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:18:27.479671 18823 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.479971 17419 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.480545 18803 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:27.483361 18658 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.669s	user 0.009s	sys 0.009s
I20260709 13:18:27.483646 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:27.483997 18658 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.484556 17410 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.489890 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:27.489912 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:27.492995 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:27.494194 18794 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.495235 18830 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.411423 18719 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.495494 18818 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.496515 18719 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.499185 16587 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:27.503192 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:27.505999 18816 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:27.507128 18597 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:27.507625 18597 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.507992 18597 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.507722 18648 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:27.509122 18803 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.509584 18803 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:27.510720 18803 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.512612 18803 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.514055 18699 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.515245 18794 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:27.525913 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 90589.5us
I20260709 13:18:27.514909 18833 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:27.517035 17411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 378) took 2947 ms (client timeout 1518 ms). Trace:
I20260709 13:18:27.517242 18651 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:18:27.519907 18546 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:27.520859 18866 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:27.520916 16595 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 1. Current term is 3. Ops: []
I20260709 13:18:27.521183 17414 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:27.521705 18738 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.522397 18658 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.522780 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:27.523517 18804 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:27.524432 18796 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:27.524677 16666 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 2.
I20260709 13:18:27.525394 18559 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:27.521350 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 134) took 1460 ms. Trace:
I20260709 13:18:27.535239 16591 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:27.556929 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.557742 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:27.558086 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.561465 18764 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.563390 18559 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:27.563812 18559 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:27.564783 18559 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.554080 16590 rpcz_store.cc:276] 0709 13:18:26.060669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:26.061523 (+   854us) service_pool.cc:224] Handling call
0709 13:18:27.223089 (+1161566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.313868 (+ 90779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.521155 (+207287us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000d98d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.521296 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670176000}
I20260709 13:18:27.566066 18559 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:27.566146 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:27.567044 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.567169 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.567907 16591 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:27.568868 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.569942 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:27.570545 16591 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:27.570437 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.571300 16589 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:27.572212 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:27.572769 16589 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.567055 16592 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.561602 18804 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:18:27.579515 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56874.6us
I20260709 13:18:27.587414 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:27.587693 16664 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.591336 16590 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.577863 18804 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:27.593559 16741 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:27.588603 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.601617 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:27.602468 16596 raft_consensus.cc:2376] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5. Re-sending same reply.
I20260709 13:18:27.603426 18796 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:27.605038 16666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 972) took 3009 ms (client timeout 2020 ms). Trace:
W20260709 13:18:27.607901 16666 rpcz_store.cc:269] 0709 13:18:24.595843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.597162 (+  1319us) service_pool.cc:224] Handling call
0709 13:18:27.604774 (+3007612us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b5400346a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.604919 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5875011968}
I20260709 13:18:27.553987 18738 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:27.543114 17411 rpcz_store.cc:269] 0709 13:18:24.569391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:24.572501 (+  3110us) service_pool.cc:224] Handling call
0709 13:18:26.911176 (+2338675us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfeee 0000000000375939 00007f3284b331e0 00007f3284bfc0c4 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:27.516750 (+605574us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400567054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.516936 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698216576}
I20260709 13:18:27.637046 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:27.640022 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.608099 18796 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:27.645402 16596 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:27.639376 18843 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:27.646131 16592 raft_consensus.cc:2376] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5. Re-sending same reply.
I20260709 13:18:27.589665 18830 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:18:27.661108 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:27.661720 18780 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:27.662889 18556 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.649412 18796 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.667165 18544 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:27.669858 16708 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.621s (SENT)
I20260709 13:18:27.671021 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:27.671247 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.674566 18842 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:18:27.676913 17413 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:27.695325 18843 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.697397 18806 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.697862 18843 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.678449 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.679253 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.685479 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:27.689244 18870 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:18:27.689525 16595 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:27.690088 18764 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:27.689811 18780 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.693320 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.700140 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:27.686689 18544 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.711089 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.619575 18651 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:18:27.721736 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.722646 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.723260 16666 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:27.726835 16664 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:27.698762 18597 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.714918 18544 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:27.729719 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 176450us
I20260709 13:18:27.734184 16592 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:27.719822 18830 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:27.748319 16595 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:27.749315 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 163752us
I20260709 13:18:27.752743 18816 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.752935 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.753230 18816 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.754263 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.733906 18803 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:27.756150 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.735278 18699 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:27.758170 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88366.7us
I20260709 13:18:27.759421 16588 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:27.740475 18240 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.737710 17410 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:27.713606 18764 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.759042 18648 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.766904 18866 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.767197 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.767271 18648 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:27.767719 18866 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.768886 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.768748 18648 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.769490 16745 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:27.770736 16593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1701) took 2522 ms (client timeout 1762 ms). Trace:
I20260709 13:18:27.771327 16594 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.747799 18833 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.773882 16588 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:27.774338 16595 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:27.771186 16593 rpcz_store.cc:269] 0709 13:18:25.247862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:25.343084 (+ 95222us) service_pool.cc:224] Handling call
0709 13:18:27.770471 (+2427387us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b540023afd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:27.770646 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4166449536}
I20260709 13:18:27.773728 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:27.775367 18240 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:18:27.779284 17414 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:27.781057 18740 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:27.781337 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.754339 18816 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:27.786605 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 175563us
I20260709 13:18:27.788094 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.790324 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.793167 16595 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:27.791384 16594 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.757521 18546 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:18:27.797987 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.799227 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.799928 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.800616 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:27.795013 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.741290 18804 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.801764 17415 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:27.790958 18240 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:27.797021 18343 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:27.843899 16740 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:27.848304 16739 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.845660 16737 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
W20260709 13:18:27.852200 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.849395 16741 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.848781 16589 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:27.853679 16668 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:27.853958 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.857280 16595 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:27.847095 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.860023 16671 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:27.839196 18738 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.864228 16743 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.868206 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:27.867823 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2241) took 1040 ms. Trace:
I20260709 13:18:27.863189 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.873787 16589 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:27.847020 18434 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.895800 16740 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:27.896723 16634 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:27.897174 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.898967 18651 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:27.899433 18651 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:27.899183 18843 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:27.899823 18651 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.900322 18843 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:27.901468 18843 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:27.903352 18738 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:18:27.902817 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:27.916922 18866 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:27.884475 18597 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.862619 18343 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:27.919049 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:27.891393 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.922657 18738 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:27.891261 18764 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.922808 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.904186 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:27.927913 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.928468 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:27.929258 16740 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:27.905524 18609 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:27.930697 18742 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.905617 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.931806 16670 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:27.905879 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:27.907011 18758 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.933879 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:27.908614 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:27.913039 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2246) took 1084 ms. Trace:
I20260709 13:18:27.939961 16591 rpcz_store.cc:276] 0709 13:18:26.828365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.588426 (+760061us) service_pool.cc:224] Handling call
0709 13:18:27.601562 (+ 13136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.602011 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.602052 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.602612 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:18:27.764019 (+161407us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:27.764062 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.764132 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:27.912822 (+148690us) raft_consensus.cc:1685] finished
0709 13:18:27.912848 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:27.912978 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5081,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":27940,"thread_start_us":161024,"threads_started":1,"wal-append.queue_time_us":308309}
W20260709 13:18:27.892429 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47447.6us
I20260709 13:18:27.913092 18693 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.942574 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:27.926734 16743 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:27.917240 18830 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:27.943661 18693 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:27.943971 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.944245 18755 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:27.903396 18818 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:27.903992 18556 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:27.913611 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:27.872776 16587 rpcz_store.cc:276] 0709 13:18:26.826993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.382340 (+555347us) service_pool.cc:224] Handling call
0709 13:18:27.382844 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.499709 (+116865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:27.867610 (+367901us) spinlock_profiling.cc:243] Waited 98.4 ms on lock 0x7b54001fc2d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:27.867767 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226313088}
I20260709 13:18:27.918027 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:27.905165 17410 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:27.892918 18816 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:27.846529 18240 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:27.930330 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.957901 16739 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:27.926334 18796 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:27.961679 16740 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.968003 16670 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:27.967162 18806 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:27.972357 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:27.972489 17415 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:27.973675 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:27.973830 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 112348us
I20260709 13:18:27.953987 18818 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.954387 18240 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:27.954761 18556 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:27.923168 18738 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:27.983083 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:27.968628 18866 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:27.997763 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.015568 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:28.015689 18556 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.016889 18556 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.017359 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:28.018570 18826 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:28.020661 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:28.021787 18864 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:28.023233 18434 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:28.022616 18235 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:18:28.025094 18840 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:28.035125 18745 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:28.043635 16745 raft_consensus.cc:2410] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:27.975023 18343 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:27.965113 18740 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:18:27.979511 18806 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:27.974648 18780 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:28.080430 16740 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:28.035243 18804 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.081703 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:27.989773 18738 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:27.987404 18764 raft_consensus.cc:2764] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:27.968358 18742 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
W20260709 13:18:28.073607 18559 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.507s	user 0.014s	sys 0.000s
I20260709 13:18:28.080360 16741 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:27.974188 18823 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:18:28.098011 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:28.106105 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 127223us
I20260709 13:18:28.110639 16667 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:28.089159 18803 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:28.099407 18719 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.000s
I20260709 13:18:28.113171 18780 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:28.122388 16671 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:28.098731 18794 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.131186 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 93043.8us
I20260709 13:18:28.120272 18544 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.133496 16670 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:28.141506 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:28.150498 16745 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:28.136581 18832 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:28.153241 16740 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:28.158603 16664 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:28.170181 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:28.146803 18755 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:28.172501 18648 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.161275 18780 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.172986 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:28.173521 16666 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:18:28.174373 18882 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:18:28.156383 18832 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:28.197121 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:28.135459 18803 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.174666 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:28.180414 18818 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.188910 18866 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:28.189713 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:28.191551 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.194008 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:28.204800 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:28.134747 18794 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:28.204989 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48742.8us
I20260709 13:18:28.193722 18547 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:28.216105 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:28.218448 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:28.225962 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:28.254815 16590 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 13:18:28.255441 16590 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:28.222505 16594 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:28.206943 18796 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:28.268925 16665 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:28.270041 16588 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:28.162678 18651 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:28.207147 18434 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:28.220679 18547 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:28.279440 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58944.1us
W20260709 13:18:28.273813 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 251766us
I20260709 13:18:28.192152 18806 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:28.288861 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 114349us
I20260709 13:18:28.273313 18711 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:28.296197 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 80199.7us
I20260709 13:18:28.297345 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:28.282886 18693 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:28.267726 18780 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:28.306298 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.316617 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.318682 18758 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:28.320070 18758 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:28.320458 18758 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:28.321420 18758 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.322728 18651 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:28.322973 18745 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.323576 18745 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:28.325968 18745 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:28.315004 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 91493.9us
I20260709 13:18:28.322642 18758 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:28.312999 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:28.300580 18711 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:28.309394 18609 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:28.309952 18693 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:28.341275 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.343182 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:28.347158 18544 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:28.348954 18807 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:28.349653 18807 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:28.350140 18807 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:28.366106 18794 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:28.369051 18796 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:28.370296 18434 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:28.371240 18884 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:18:28.312036 18832 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:28.382321 18839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.030s	sys 0.037s
W20260709 13:18:28.382764 18839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.030s	sys 0.037s
W20260709 13:18:28.392071 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62810.4us
W20260709 13:18:28.395591 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 57873.8us
I20260709 13:18:28.397962 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:28.398844 16588 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:28.400887 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:28.399624 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 283617us
I20260709 13:18:28.336921 18711 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:28.375705 18693 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:28.414745 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:28.416227 16595 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:28.417268 16587 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 13:18:28.430085 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:28.436878 16665 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:28.441423 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:28.442291 16587 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:28.410717 18794 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:18:28.428395 18546 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:18:28.482903 18816 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:18:28.513855 18827 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.484459 18889 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:18:28.485453 18681 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:18:28.434571 18343 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:18:28.460997 18826 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:28.490049 18812 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:18:28.491202 18780 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:28.438580 18699 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:18:28.494985 18839 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:28.555949 18742 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:28.556427 18742 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:28.557060 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 116047us
I20260709 13:18:28.558102 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:28.504966 18738 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:28.505944 16588 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:28.507887 16587 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:28.489542 16589 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 2. (term mismatch)
I20260709 13:18:28.458673 18803 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:28.438725 18804 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:18:28.550140 18827 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:28.551802 16665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1012) took 1389 ms (client timeout 1792 ms). Trace:
I20260709 13:18:28.429415 18547 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.571242 16589 pending_rounds.cc:85] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 1
W20260709 13:18:28.599946 16665 rpcz_store.cc:269] 0709 13:18:27.162505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.323818 (+161313us) service_pool.cc:224] Handling call
0709 13:18:28.551591 (+1227773us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b540001b654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:28.551706 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170908032}
I20260709 13:18:28.610817 16589 pending_rounds.cc:107] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:18:28.611203 16589 raft_consensus.cc:2889] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:28.657224 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2256) took 1115 ms. Trace:
I20260709 13:18:28.657799 16590 rpcz_store.cc:276] 0709 13:18:27.492150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.565852 (+ 73702us) service_pool.cc:224] Handling call
0709 13:18:28.607644 (+1041792us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b5400052654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:28.607810 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40494336}
I20260709 13:18:28.612958 16592 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:28.622130 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 61882.6us
W20260709 13:18:28.556005 18830 consensus_meta.cc:287] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.000s
I20260709 13:18:28.662667 16593 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:28.527235 18546 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
W20260709 13:18:28.560796 18864 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.014s	sys 0.011s
I20260709 13:18:28.609303 18780 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:18:28.698374 18240 consensus_meta.cc:287] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.691s	user 0.008s	sys 0.003s
I20260709 13:18:28.690577 18830 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:28.703558 18880 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:18:28.693729 18794 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:28.747157 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 307258us
I20260709 13:18:28.748040 16739 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:28.717798 18864 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.014s	sys 0.012s
I20260709 13:18:28.702214 17410 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
W20260709 13:18:28.751855 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 315436us
I20260709 13:18:28.752770 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:28.557420 18742 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:28.755486 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47731.3us
W20260709 13:18:28.763928 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 339429us
I20260709 13:18:28.765319 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:28.773327 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:28.736430 18240 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:28.810217 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.822183 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:28.742093 18794 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.827579 16665 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:28.836575 16739 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:28.772168 18434 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.787175 18742 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:28.846910 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81644.5us
I20260709 13:18:28.851179 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:28.837707 18745 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:28.901027 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:28.908102 16737 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:28.886236 18912 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:28.914562 18915 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.920125 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:28.915442 18693 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.506s	user 0.015s	sys 0.006s
I20260709 13:18:28.905970 18738 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:28.905820 18740 raft_consensus.cc:670] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } [suppressed 3 similar messages]
I20260709 13:18:28.921535 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:28.848363 18434 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:28.926293 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:28.925132 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74246.5us
W20260709 13:18:28.935796 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.935739 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:28.940781 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:28.919193 18912 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:28.945094 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:28.923821 18681 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.954826 16741 raft_consensus.cc:2376] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6. Re-sending same reply.
W20260709 13:18:28.959444 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:28.958089 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:28.943878 18807 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.002s
I20260709 13:18:28.963083 16740 raft_consensus.cc:2376] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5. Re-sending same reply.
I20260709 13:18:28.941345 18559 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:28.964728 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:28.965075 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:28.973794 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:28.947830 18898 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:28.943256 18827 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:28.954278 18648 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.757s	user 0.007s	sys 0.004s
I20260709 13:18:28.975857 16740 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [3.3-4.4]
I20260709 13:18:28.979933 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:28.930107 18693 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:28.985333 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:28.986814 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:28.970434 18812 raft_consensus.cc:2764] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:28.969805 18864 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:28.990780 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1026) took 1003 ms. Trace:
I20260709 13:18:28.981496 18796 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:28.926541 18915 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:28.993263 16741 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:28.996176 16745 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:18:28.998279 16741 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:18:28.964305 18681 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.001348 17416 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:29.001550 18917 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:28.970506 18807 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:28.979683 18918 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:29.000064 18711 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.002s
I20260709 13:18:28.991250 16662 rpcz_store.cc:276] 0709 13:18:27.987225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.007793 (+ 20568us) service_pool.cc:224] Handling call
0709 13:18:28.023410 (+ 15617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.152895 (+129485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.152924 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.192816 (+ 39892us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.657148 (+464332us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:28.657191 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.657258 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:28.990563 (+333305us) raft_consensus.cc:1685] finished
0709 13:18:28.990581 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:28.990717 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17482,"raft.queue_time_us":58096,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5883,"thread_start_us":508370,"threads_started":2,"wal-append.queue_time_us":626260}
I20260709 13:18:28.996831 18240 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.019058 16745 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:28.997170 18796 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:29.026640 16741 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:29.006532 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:29.032765 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:29.033532 17419 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:29.006990 18864 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.038025 18864 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.040473 18807 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:29.045244 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:29.051369 18900 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
I20260709 13:18:29.052134 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:29.053013 17420 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:29.054430 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.055480 16744 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:29.055341 18918 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.057005 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.058007 18918 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:29.054822 18906 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.203s	user 0.001s	sys 0.001s
I20260709 13:18:29.060708 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.061743 16744 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:29.042722 18693 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:29.063362 18903 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:18:29.065738 18681 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.003928 18917 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.068295 18917 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.072153 16745 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:18:29.062605 18758 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.001s
I20260709 13:18:29.076949 18755 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:29.077627 18755 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.083798 16666 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:29.085163 17413 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:29.086575 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.089253 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:29.090606 16666 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:29.030349 18794 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.098243 18915 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:29.099012 18905 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:18:29.024801 18711 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.102283 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.104096 18711 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.114801 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 218) took 1221 ms. Trace:
I20260709 13:18:29.104193 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.105307 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:29.115813 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 342546us
I20260709 13:18:29.118057 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.105719 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.116070 18826 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:29.070261 18796 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:29.119361 18823 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:29.115813 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.119896 18823 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:29.120330 18823 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.121665 18830 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:29.122507 18830 consensus_queue.cc:1243] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:29.107569 18434 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:28.999814 18912 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.123096 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.123966 18912 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.124153 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:29.124259 18898 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:29.125321 18898 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:29.125720 18648 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:29.126048 18898 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:29.126139 18648 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:29.127092 18826 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:29.127539 18826 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:29.127941 18826 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.129088 18648 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.130388 18648 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.131150 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1729) took 2130 ms. Trace:
I20260709 13:18:29.131409 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:29.131553 16592 rpcz_store.cc:276] 0709 13:18:27.153622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.738047 (+584425us) service_pool.cc:224] Handling call
0709 13:18:28.607932 (+869885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.671785 (+ 63853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.130968 (+459183us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b5400052650. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:29.131094 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1999458176}
I20260709 13:18:29.132324 18827 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:29.132730 18827 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:29.133093 18827 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.116711 16743 rpcz_store.cc:276] 0709 13:18:27.892926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.944991 (+ 52065us) service_pool.cc:224] Handling call
0709 13:18:27.945349 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:27.945614 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:28.405087 (+459473us) log.cc:844] Serialized 14 byte log entry
0709 13:18:28.769872 (+364785us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.114197 (+344325us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b180043b760. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:18:29.114385 (+   188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.114442 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:29.114492 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.114564 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.114710 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":308946,"prepare.queue_time_us":399134,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":88,"raft.queue_time_us":353884,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":6672,"spinlock_wait_cycles":35599360,"thread_start_us":761421,"threads_started":2,"wal-append.queue_time_us":486277}
I20260709 13:18:29.137023 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:29.109344 18745 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:29.115320 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.098722 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.150758 16737 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:29.152040 16742 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:29.156867 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.157645 16741 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:29.159554 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:29.161201 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.126605 18898 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:29.163496 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.163659 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.164260 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.164426 18843 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:29.167418 16663 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:29.169243 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.187829 17416 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:29.190833 16668 raft_consensus.cc:2410] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:18:29.189613 16738 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 2.
I20260709 13:18:29.195461 16743 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:29.198227 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:29.198271 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54780.1us
I20260709 13:18:29.200670 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:29.201507 17415 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:29.199226 17420 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
W20260709 13:18:29.210753 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 225473us
I20260709 13:18:29.156419 18745 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.211750 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:29.213475 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44489.1us
I20260709 13:18:29.214128 16671 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:29.218120 16738 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 13:18:29.219137 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.219535 17416 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:29.223208 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.223984 16745 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.231237 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.188278 18843 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:29.218520 16738 pending_rounds.cc:85] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 5
I20260709 13:18:29.238072 16738 pending_rounds.cc:107] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:18:29.238467 16738 raft_consensus.cc:2889] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:29.238903 16741 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:29.220554 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:29.235142 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.217317 17411 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:29.245594 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.217010 18807 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.246341 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.245009 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.251369 16671 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:29.256826 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.244576 18758 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:29.273811 16738 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:29.265571 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48626.3us
I20260709 13:18:29.244077 18559 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.256260 18807 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.279912 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.280627 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.244627 18830 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.281251 18758 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:29.283512 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:29.295558 16745 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:29.231156 18796 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.303558 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:29.249536 18780 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:29.305755 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.577s (SENT) [suppressed 30 similar messages]
I20260709 13:18:29.301728 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.302907 18830 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.304512 16667 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:29.249173 18895 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:29.271787 18806 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.273782 18719 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.315968 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:29.312704 18434 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:29.310703 18235 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:29.325426 18758 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.324254 18742 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:29.337371 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:29.326633 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 444) took 1344 ms. Trace:
I20260709 13:18:29.311914 18895 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.358229 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.317821 18719 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:29.316144 18780 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:29.320349 18806 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.337903 18235 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:29.368184 18742 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:29.381382 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 902) took 1432 ms. Trace:
I20260709 13:18:29.381664 18742 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:29.382071 16738 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:29.382027 16739 rpcz_store.cc:276] 0709 13:18:27.948736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.959655 (+ 10919us) service_pool.cc:224] Handling call
0709 13:18:29.381110 (+1421455us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54002b8ed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:29.381268 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757144832}
I20260709 13:18:29.341522 17410 rpcz_store.cc:276] 0709 13:18:27.982263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.006659 (+ 24396us) service_pool.cc:224] Handling call
0709 13:18:28.008991 (+  2332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.762837 (+753846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.326231 (+563394us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54005492d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:29.326515 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523498752}
I20260709 13:18:29.389319 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.370837 18240 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:18:29.390552 16739 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:29.392091 18916 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:29.392822 18916 raft_consensus.cc:2764] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:18:29.393539 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:29.394244 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 415385us
I20260709 13:18:29.394979 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:29.360266 18559 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:29.395316 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.399667 16739 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:29.327641 18843 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.401417 18843 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:29.401985 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.402693 16662 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:29.403539 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.382170 18742 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.404465 18830 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:29.404853 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.405880 16662 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:29.408023 18927 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:29.409373 18927 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.410430 18927 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.406831 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.366091 18434 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:29.425158 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55862.5us
I20260709 13:18:29.426606 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.428390 18866 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.428519 18864 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:18:29.433434 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 129088us
I20260709 13:18:29.429065 18866 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.437281 16595 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:29.411514 18796 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.438513 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:29.439707 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 228062us
I20260709 13:18:29.487898 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:29.467509 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.408212 18812 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:29.491361 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52987.8us
I20260709 13:18:29.374378 18597 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:18:29.431100 18434 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:29.510609 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1033) took 1162 ms. Trace:
I20260709 13:18:29.512140 16670 rpcz_store.cc:276] 0709 13:18:28.342280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.342505 (+   225us) service_pool.cc:224] Handling call
0709 13:18:28.362517 (+ 20012us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.362924 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.362958 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.363460 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.700852 (+337392us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:28.700894 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.700963 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.368398 (+667435us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b240010ddc8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327c97d9c4 00007f327c97bacb 00007f327cb36dc9 00007f327cb36a55 00007f327cb35fb0 00007f327f7ac488 00007f327f7acbe8 00007f327f784398 00007f327f77f76a 00007f327f77ebb6
0709 13:18:29.468192 (+ 99794us) raft_consensus.cc:1685] finished
0709 13:18:29.468223 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.504323 (+ 36100us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":145,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":28372224,"thread_start_us":337000,"threads_started":1,"wal-append.queue_time_us":685530}
I20260709 13:18:29.435055 18864 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:29.432041 18939 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.516954 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:29.514789 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.487455 18912 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:29.507114 18693 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:29.505677 18812 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:29.518209 18434 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:29.574817 18864 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:29.515218 18648 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:29.591876 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.403849 18235 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:29.598882 18711 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:29.423007 18839 consensus_queue.cc:1175] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "efa00eb5e7404570958fdebb6cd077e8" responder_term: 4 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: [3.2-3.2]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:29.602890 18711 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.604934 18711 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:29.605880 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 118097us
I20260709 13:18:29.580358 18807 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:29.606667 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:29.578306 18939 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.607788 18866 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:29.607861 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:29.608984 18719 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.609421 18939 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:29.609977 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:29.610368 18910 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:29.610909 18910 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.611618 18916 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:29.612344 18916 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:29.611856 18910 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:29.612931 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:29.613260 18912 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.615043 18343 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:29.615150 18918 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:29.615564 18343 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.616497 18546 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:29.616999 18546 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.616822 18648 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.617905 18546 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.619127 18648 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:29.619530 18780 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:29.609455 16595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38574
I20260709 13:18:29.621091 16595 raft_consensus.cc:606] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Received request to transfer leadership
I20260709 13:18:29.622272 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:29.582356 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.509773 18908 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:18:29.622532 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2267) took 1231 ms. Trace:
I20260709 13:18:29.624434 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:29.583830 18812 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:29.612869 18916 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:29.599078 18932 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.630566 16594 rpcz_store.cc:276] 0709 13:18:28.391489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.391630 (+   141us) service_pool.cc:224] Handling call
0709 13:18:28.404697 (+ 13067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:28.404946 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:28.758611 (+353665us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.391552 (+632941us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b18003973c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:18:29.622203 (+230651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:29.622250 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:29.622298 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.622389 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.622481 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32841,"prepare.queue_time_us":107362,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":29,"raft.queue_time_us":420442,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":34889984,"thread_start_us":773391,"threads_started":3,"wal-append.queue_time_us":244005}
I20260709 13:18:29.614315 18912 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.630014 18699 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:29.664441 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:29.615589 18806 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:29.635368 18826 consensus_meta.cc:287] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.005s
I20260709 13:18:29.663677 16738 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:29.635679 18559 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:29.677585 18818 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.688287 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.629920 18918 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:18:29.633430 18742 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.691298 16739 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:29.616518 18343 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.707109 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:29.707234 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.707686 16589 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:29.707728 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:29.697365 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 98183.1us
W20260709 13:18:29.688874 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 196513us
I20260709 13:18:29.697782 16588 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:18:29.629760 18711 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:18:29.700769 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.695744 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.696858 16738 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:29.709156 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:29.713640 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.713721 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:29.712756 16740 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:29.715139 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.715732 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.712757 16590 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:29.717556 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.709604 18597 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:29.711264 16594 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:29.708997 16588 pending_rounds.cc:85] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 4
I20260709 13:18:29.689512 18806 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:29.719471 16588 pending_rounds.cc:107] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:18:29.726439 16588 raft_consensus.cc:2889] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:29.720964 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.721094 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:29.720269 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1727) took 2730 ms. Trace:
I20260709 13:18:29.729614 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.722748 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.723232 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:29.727631 16596 rpcz_store.cc:276] 0709 13:18:26.990043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.717637 (+727594us) service_pool.cc:224] Handling call
0709 13:18:27.736766 (+ 19129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.737168 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:27.737195 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:27.763721 (+ 26526us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.036754 (+273033us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:28.279311 (+242557us) log.cc:844] Serialized 14 byte log entry
0709 13:18:29.110702 (+831391us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54000931a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb5f733 00007f327fbe4dbf 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c
0709 13:18:29.110755 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.110832 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.110904 (+    72us) raft_consensus.cc:1685] finished
0709 13:18:29.111035 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.720086 (+609051us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b540010cf50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:29.720210 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":409947,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":5330,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":20778,"spinlock_wait_cycles":283611136,"thread_start_us":423933,"threads_started":2,"wal-append.queue_time_us":302347,"wal-append.run_cpu_time_us":5258,"wal-append.run_wall_time_us":1296812}
W20260709 13:18:29.753209 18755 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.001s
I20260709 13:18:29.701578 18804 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.755429 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.756210 16667 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:29.756857 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.757496 16744 raft_consensus.cc:2410] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:29.758256 16738 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:29.862363 17420 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:29.722261 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:29.865131 16740 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:29.672235 18932 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:29.791898 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:29.873553 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.705847 18915 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:29.802762 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.756560 18934 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.877593 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1036) took 1106 ms. Trace:
I20260709 13:18:29.700870 18609 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:29.720269 18597 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:29.809168 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.813840 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:29.809938 18916 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.800432 18755 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:29.831087 18930 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:29.808434 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:29.855958 16588 raft_consensus.cc:2410] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:29.870636 17413 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:18:29.871658 18949 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:18:29.872169 18964 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:29.872370 18902 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.891s	user 0.003s	sys 0.000s
I20260709 13:18:29.686045 18826 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:29.896250 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.962178 16591 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
W20260709 13:18:29.879472 18823 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.004s
W20260709 13:18:29.877722 18745 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.000s
I20260709 13:18:29.903312 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.905547 16589 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:29.975984 16737 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:29.976166 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:29.924458 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.976754 16592 raft_consensus.cc:2376] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:29.878139 16664 rpcz_store.cc:276] 0709 13:18:28.771601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.771772 (+   171us) service_pool.cc:224] Handling call
0709 13:18:28.809340 (+ 37568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.961929 (+152589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.961958 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.979443 (+ 17485us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.167430 (+187987us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:29.167489 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.839062 (+671573us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540009d654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:29.858749 (+ 19687us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.858813 (+    64us) raft_consensus.cc:1685] finished
0709 13:18:29.858835 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:29.877530 (+ 18695us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2143,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8688,"spinlock_wait_cycles":349998976,"thread_start_us":147498,"threads_started":1,"wal-append.queue_time_us":254656}
I20260709 13:18:29.923825 17410 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:29.960999 18742 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.936064 18758 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.895736 18609 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.878528 18738 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 13:18:29.995908 16742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 215) took 2204 ms (client timeout 2937 ms). Trace:
I20260709 13:18:29.997273 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:29.997570 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.015491 16668 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:29.992942 16739 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:18:29.892688 18827 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.759s	user 0.009s	sys 0.009s
I20260709 13:18:29.990458 16664 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:18:29.980199 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 266568us
I20260709 13:18:30.074733 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.895328 18804 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.096061 18826 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.096715 18804 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.097237 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:30.098315 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:30.101608 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:30.101804 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:30.103068 18864 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.103775 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.105096 18864 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:29.963653 18866 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.092383 18755 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.092896 18796 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.108955 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:29.940640 18693 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:30.094163 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.080047 18597 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.077983 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:30.115549 16589 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 2.
W20260709 13:18:30.005867 18952 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:18:30.102614 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:29.995147 18932 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.111310 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.135897 16663 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:18:30.008049 18956 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:18:30.116272 16671 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:29.992302 18832 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:30.121459 16594 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:30.083362 16742 rpcz_store.cc:269] 0709 13:18:27.791499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.791628 (+   129us) service_pool.cc:224] Handling call
0709 13:18:29.995692 (+2204064us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54003ed5d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:29.995817 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2732650240}
I20260709 13:18:29.891996 18915 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.989766 18745 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.128039 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.006088 18959 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:29.981096 18964 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.131954 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:29.990850 18816 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.132s
I20260709 13:18:30.152266 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.133105 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:30.124327 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.121971 18939 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:29.891814 18934 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:29.983038 18823 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.253520 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:30.257210 16743 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:30.222783 18972 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:30.126011 18804 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:30.080787 18758 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.158274 16589 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:30.119942 18796 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.107739 18827 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.202535 16664 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:18:30.114622 18866 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.185102 16742 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
W20260709 13:18:30.222208 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 60147us
I20260709 13:18:30.187927 18745 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:30.159682 18843 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.001s
I20260709 13:18:30.151870 18597 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.159193 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 393) took 1205 ms. Trace:
I20260709 13:18:30.189965 16668 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:30.180456 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:30.292300 16594 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.195657 16590 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:30.198320 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:30.293962 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:30.215185 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:30.160475 18839 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:18:30.298265 16744 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:30.191176 18959 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:30.233135 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.193295 18755 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:30.233633 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:30.188494 18693 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.180009 18964 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.187033 18812 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:30.113854 18902 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:18:30.264453 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:30.274603 16745 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:30.313181 18235 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:30.313271 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48925.5us
I20260709 13:18:30.250662 18963 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:30.314061 18434 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.692s	user 0.005s	sys 0.006s
I20260709 13:18:30.314489 18963 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:30.286633 18816 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:18:30.316805 18826 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.289934 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:30.266446 18804 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.296964 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:30.271608 18972 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:30.304194 18755 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:30.268709 18806 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:30.260701 18953 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:18:30.307925 16592 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:30.308581 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:30.276633 16593 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:30.316937 18963 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.287695 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.270367 18939 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.327378 16589 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:30.315168 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:30.329244 16743 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:30.323244 16590 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:30.314592 18434 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.330915 18806 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:30.331370 18806 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:30.332305 18806 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.333524 18806 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.335793 18796 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.335872 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.336453 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 213) took 2558 ms. Trace:
I20260709 13:18:30.337128 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.337499 18895 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.338411 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:30.339912 18827 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.340190 16742 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:18:30.340602 16742 pending_rounds.cc:85] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 3
I20260709 13:18:30.340831 16742 pending_rounds.cc:107] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:30.341053 16742 raft_consensus.cc:2889] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:30.343693 18758 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.316772 18823 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.344158 18816 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:30.344941 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.345110 18939 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.336836 16736 rpcz_store.cc:276] 0709 13:18:27.766119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:27.770965 (+  4846us) service_pool.cc:224] Handling call
0709 13:18:27.771301 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:27.771867 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:28.141401 (+369534us) log.cc:844] Serialized 14 byte log entry
0709 13:18:28.756653 (+615252us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.824089 (+ 67436us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.824486 (+   397us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:28.824610 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.824799 (+   189us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:29.872050 (+1047251us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b240022adf8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327c97d9c4 00007f327c97bacb 00007f327cb36dc9 00007f327cb36a55 00007f327cb3609d 00007f327f76810e 00007f327f77f88c 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221
0709 13:18:30.318722 (+446672us) raft_consensus.cc:1685] finished
0709 13:18:30.324179 (+  5457us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.324323 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":341781,"prepare.queue_time_us":538932,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":1077,"raft.queue_time_us":5934,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":11046,"spinlock_wait_cycles":57429760,"thread_start_us":831643,"threads_started":2,"wal-append.queue_time_us":698877}
I20260709 13:18:30.290438 17414 rpcz_store.cc:276] 0709 13:18:28.953974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.970713 (+ 16739us) service_pool.cc:224] Handling call
0709 13:18:29.217464 (+246751us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfe9c 00007f327cabfa5b 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:29.243063 (+ 25599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:29.302952 (+ 59889us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.302993 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:29.303620 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.577738 (+274118us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:29.582566 (+  4828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:29.593466 (+ 10900us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.130563 (+537097us) raft_consensus.cc:1685] finished
0709 13:18:30.130589 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.157558 (+ 26969us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16681,"raft.queue_time_us":29206,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4433,"spinlock_wait_cycles":5507840,"thread_start_us":184108,"threads_started":1,"wal-append.queue_time_us":393641}
W20260709 13:18:30.346897 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57043.9us
I20260709 13:18:30.349237 18912 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.349642 18912 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.349838 18982 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.350052 18912 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.350463 18982 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.322738 18804 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.352751 18982 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.353395 16592 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.323958 18755 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:30.363224 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48140.5us
I20260709 13:18:30.364662 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:30.311612 18609 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.367893 16594 raft_consensus.cc:2410] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:30.367898 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.374539 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.375578 16594 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 13:18:30.380316 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:30.381417 16743 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:30.385428 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:30.385989 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92803.6us
I20260709 13:18:30.386605 16589 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:30.388551 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.387081 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:30.390179 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 495977us
I20260709 13:18:30.366340 18915 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.392189 16594 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:30.359355 18959 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:30.396263 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:30.387812 18818 consensus_meta.cc:287] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.001s
I20260709 13:18:30.383142 18742 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.403455 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.404122 16594 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:30.404861 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.405321 16594 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:30.406095 16666 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:30.388674 18827 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.408668 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.409855 16589 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:30.406899 18975 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.167s	user 0.001s	sys 0.001s
I20260709 13:18:30.415459 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:30.418495 18978 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:18:30.420310 16589 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:30.406775 18968 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.325104 18898 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.427402 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1039) took 1649 ms. Trace:
I20260709 13:18:30.410619 18823 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:30.414793 18970 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:18:30.440531 16589 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:30.406705 18839 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:18:30.432140 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:30.432431 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:30.434404 18974 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:18:30.413931 18693 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:30.413322 18827 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:30.432826 16669 rpcz_store.cc:276] 0709 13:18:28.777659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.777812 (+   153us) service_pool.cc:224] Handling call
0709 13:18:28.828769 (+ 50957us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:28.907466 (+ 78697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:28.907507 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:28.915870 (+  8363us) log.cc:844] Serialized 26 byte log entry
0709 13:18:28.994045 (+ 78175us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:28.994090 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:28.994161 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.419837 (+1425676us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b240010ddc8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327c97d9c4 00007f327c97bacb 00007f327cb36dc9 00007f327cb36a55 00007f327cb35fb0 00007f327f7ac488 00007f327f7acbe8 00007f327f784398 00007f327f77f76a 00007f327f77ebb6
0709 13:18:30.423182 (+  3345us) raft_consensus.cc:1685] finished
0709 13:18:30.423203 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:30.427338 (+  4135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60230,"raft.queue_time_us":71265,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":6047,"spinlock_wait_cycles":56806272,"thread_start_us":44386,"threads_started":1,"wal-append.queue_time_us":113346}
I20260709 13:18:30.449311 16665 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:30.443300 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:30.527230 16592 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:30.472762 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2208) took 1202 ms. Trace:
I20260709 13:18:30.448333 16589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38574
I20260709 13:18:30.533159 16589 raft_consensus.cc:606] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Received request to transfer leadership
I20260709 13:18:30.505992 18806 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:30.473230 16737 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:30.535202 18920 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.535903 18920 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.449311 18758 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:18:30.431282 18968 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.450407 18823 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:30.542171 18827 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:30.543021 18981 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:18:30.543860 18826 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:30.544374 18826 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.440707 18963 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 13:18:30.441797 18987 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.103s
I20260709 13:18:30.543346 18823 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.531647 16745 rpcz_store.cc:276] 0709 13:18:29.269846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.301417 (+ 31571us) service_pool.cc:224] Handling call
0709 13:18:30.472548 (+1171131us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b5400306454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:30.472699 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224929664}
I20260709 13:18:30.504069 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:30.549594 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 134198us
W20260709 13:18:30.551239 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 108005us
I20260709 13:18:30.554358 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:30.558720 18832 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.563849 18693 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:30.465905 18699 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:30.564816 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 132757us
I20260709 13:18:30.570362 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:30.571185 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:30.414620 18972 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:30.573496 16636 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Timed out: RequestConsensusVote RPC to 127.15.29.4:41455 timed out after 1.518s (SENT)
I20260709 13:18:30.467362 18953 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.506567 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:30.574687 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:30.552949 18920 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.575770 18699 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.576727 18988 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:30.463903 18980 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:30.603255 18980 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:30.604228 18980 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.606719 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:30.607048 16743 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:30.607687 18839 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:30.609205 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.609236 18840 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:30.610000 18818 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:30.610488 18818 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:30.575717 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.442553 18939 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.138s
I20260709 13:18:30.502170 18597 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.076s
I20260709 13:18:30.578747 18953 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:30.581144 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:30.631644 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50588.5us
W20260709 13:18:30.582311 18976 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.291s	user 0.001s	sys 0.001s
I20260709 13:18:30.582845 18996 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:18:30.635975 18996 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:30.636477 18996 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:30.534089 18806 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.589350 18968 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.464345 18812 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:30.513420 18908 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:18:30.640762 18755 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.643754 18806 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.610656 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.564435 18693 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:30.630159 18920 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:18:30.618176 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44429.7us
W20260709 13:18:30.596995 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 93043.5us
I20260709 13:18:30.627444 18988 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:30.659760 18988 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.660192 18988 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.660392 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:30.660811 18693 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.662046 16587 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:30.662336 18693 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:30.648092 18953 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.663213 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:30.663821 16592 raft_consensus.cc:2376] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5. Re-sending same reply.
I20260709 13:18:30.667724 18896 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:30.668352 18896 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:30.669261 18896 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.669756 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:30.670627 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.671804 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.672405 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:30.643582 18912 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.628929 18742 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:30.673225 16736 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:30.611027 18816 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:18:30.672370 16589 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:30.646116 18827 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:30.655045 18908 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:30.655498 18806 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.681854 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:30.689496 16740 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:30.654844 18920 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:30.691272 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:30.659765 18755 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:30.693970 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:30.703289 16589 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:30.694114 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.611445 18818 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.696221 18963 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:30.634778 18804 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.679757 18719 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.699544 16736 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:18:30.723358 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.699901 18681 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.700352 18920 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.705729 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:30.749405 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 43788.6us
I20260709 13:18:30.750489 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:30.719226 19010 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:30.691190 18908 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.699220 18806 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:30.724042 16478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 99384.2us
I20260709 13:18:30.725500 18912 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.727214 18921 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:30.715276 18742 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:30.791818 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.875692 16594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:38574
I20260709 13:18:30.876492 16594 raft_consensus.cc:606] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Received request to transfer leadership
I20260709 13:18:30.771664 18832 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:30.807087 18908 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.812150 18920 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:30.812521 18742 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:30.894677 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 211801us
I20260709 13:18:30.702755 18964 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.750488 18963 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.749643 18719 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.908305 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:30.886132 18832 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:30.751646 18996 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:30.817592 18921 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:30.902410 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.901239 18742 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:30.938695 18832 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:30.905583 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:30.948596 18991 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:18:30.947522 18989 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:30.898303 18826 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.988094 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:31.013912 18832 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:30.905578 18864 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
W20260709 13:18:30.949337 18958 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:18:30.940829 18755 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:30.978549 19000 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:18:30.978116 19011 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:31.024207 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.979198 18996 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:30.997813 18912 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.020745 17409 raft_consensus.cc:2376] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4. Re-sending same reply.
I20260709 13:18:31.021040 18989 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:31.021440 18827 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:31.030335 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:30.988258 19009 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:18:31.027595 18755 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:31.029480 18980 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.059052 18998 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.051115 19011 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.069296 18963 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:31.053225 18996 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.071323 18921 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:31.071835 18804 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:31.033331 18719 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:31.035274 19007 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.030321 19017 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:31.077224 17409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 398) took 2082 ms (client timeout 1518 ms). Trace:
I20260709 13:18:31.096378 19011 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:31.097148 18996 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:31.100195 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 112209us
I20260709 13:18:31.102700 19021 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:31.106685 19007 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:31.048939 18912 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:31.103252 18806 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:31.127154 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:31.127538 18989 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.130005 18547 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:31.133805 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:31.135145 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:31.136648 16742 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:31.112730 18827 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:31.092037 18980 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.138689 16742 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:31.126369 19011 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:31.092415 18998 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.143251 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:31.102041 17409 rpcz_store.cc:269] 0709 13:18:28.970042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:28.974526 (+  4484us) service_pool.cc:224] Handling call
0709 13:18:30.897189 (+1922663us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:31.047871 (+150682us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5485440}
I20260709 13:18:31.167398 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:31.171944 18912 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:31.172607 18699 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.002s
I20260709 13:18:31.147071 18755 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.155337 18827 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:31.184371 18699 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:31.186112 18827 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:31.186321 18699 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:31.187745 18988 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 13:18:31.146910 18964 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:31.152339 19011 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:31.189517 18953 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:18:31.206785 18951 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:31.213860 18951 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:31.188181 18988 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.212894 18953 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.245705 16736 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:31.227236 19011 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:31.249346 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:31.250034 16745 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:31.262619 18742 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:31.264571 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 910) took 1598 ms. Trace:
I20260709 13:18:31.264997 16744 rpcz_store.cc:276] 0709 13:18:29.666282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.870553 (+204271us) service_pool.cc:224] Handling call
0709 13:18:29.870842 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:30.298918 (+428076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.264306 (+965388us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b54003d0754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:31.264506 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":978401024}
W20260709 13:18:31.275295 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:31.291740 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 146929us
W20260709 13:18:31.292052 18681 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.002s
I20260709 13:18:31.302610 18681 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:31.303220 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1702) took 2110 ms. Trace:
I20260709 13:18:31.303665 16666 rpcz_store.cc:276] 0709 13:18:29.193012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.203957 (+ 10945us) service_pool.cc:224] Handling call
0709 13:18:31.303035 (+2099078us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54000f37d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:31.303156 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2681768704}
W20260709 13:18:31.306705 16708 leader_election.cc:336] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.577s (SENT)
W20260709 13:18:32.091750 16745 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.841s	user 0.034s	sys 0.026s
W20260709 13:18:32.091991 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 785271us
W20260709 13:18:31.834782 16738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.011s	sys 0.015s
W20260709 13:18:32.092979 16738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.016s	sys 0.018s
W20260709 13:18:31.719239 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:31.454962 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:31.369972 16739 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 913) took 1702 ms (client timeout 1673 ms). Trace:
W20260709 13:18:31.857122 16742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.007s	sys 0.027s
W20260709 13:18:32.095479 16742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.010s	sys 0.031s
I20260709 13:18:31.376888 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 153) took 1616 ms. Trace:
I20260709 13:18:32.098146 16593 rpcz_store.cc:276] 0709 13:18:29.760173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.103495 (+343322us) service_pool.cc:224] Handling call
0709 13:18:31.376676 (+1273181us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400190fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:31.376827 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395959552}
I20260709 13:18:31.377667 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2204) took 2109 ms. Trace:
W20260709 13:18:32.094758 16739 rpcz_store.cc:269] 0709 13:18:29.667045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.704118 (+ 37073us) service_pool.cc:224] Handling call
0709 13:18:31.369639 (+1665521us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400059e54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:31.369863 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541970816}
I20260709 13:18:32.092166 16745 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:31.638324 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:32.126329 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 833742us
I20260709 13:18:32.126745 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.127446 17420 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:32.126302 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 239) took 1002 ms. Trace:
I20260709 13:18:32.128808 16740 rpcz_store.cc:276] 0709 13:18:31.123576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.124916 (+1001340us) service_pool.cc:224] Handling call
0709 13:18:32.125411 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.125756 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:32.125804 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:32.125867 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:32.125922 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.126002 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.126191 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:31.627799 16621 heartbeater.cc:646] Failed to heartbeat to 127.15.29.62:37247 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.15.29.62:37247 timed out after 15.000s (SENT)
I20260709 13:18:32.099593 16741 rpcz_store.cc:276] 0709 13:18:29.268722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.278568 (+  9846us) service_pool.cc:224] Handling call
0709 13:18:29.279010 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:29.583712 (+304702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.591596 (+  7884us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:29.601848 (+ 10252us) log.cc:844] Serialized 26 byte log entry
0709 13:18:29.810905 (+209057us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:29.810953 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.377173 (+1566220us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b5400049054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:31.377226 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.377275 (+    49us) raft_consensus.cc:1685] finished
0709 13:18:31.377290 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:31.377574 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89894,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":68980096,"thread_start_us":448772,"threads_started":2,"wal-append.queue_time_us":536063,"wal-append.run_cpu_time_us":3692,"wal-append.run_wall_time_us":1425729}
I20260709 13:18:32.136173 17420 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:31.374191 19005 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.471s	user 0.000s	sys 0.002s
W20260709 13:18:31.376080 19008 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
W20260709 13:18:31.321138 18932 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.135s	user 0.003s	sys 0.021s
I20260709 13:18:32.175634 16744 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:31.792804 18681 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.172268 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:31.439492 18745 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:31.506274 16493 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:32.092459 19011 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.181931 16594 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:31.382292 18597 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:31.633965 19015 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.600s	user 0.000s	sys 0.002s
I20260709 13:18:32.176365 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 238) took 1053 ms. Trace:
I20260709 13:18:31.386677 18544 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.179925 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:32.211629 16744 rpcz_store.cc:276] 0709 13:18:31.123300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.124947 (+1001647us) service_pool.cc:224] Handling call
0709 13:18:32.175446 (+ 50499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:32.176106 (+   660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.176310 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:32.175731 18932 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.989s	user 0.007s	sys 0.041s
I20260709 13:18:32.124945 18699 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:32.111959 18823 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.110945 18998 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:32.135557 18908 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.234s	user 0.032s	sys 0.041s
I20260709 13:18:32.138032 18738 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:32.173475 19016 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.124758 18980 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.128846 18742 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.173245 19007 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.209909 18597 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.178607 18434 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.189422 18745 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:32.223937 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 131024us
W20260709 13:18:32.225318 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 207999us
I20260709 13:18:32.187870 18996 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.139360 19010 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:32.227584 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 589332us
I20260709 13:18:32.184352 18681 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.230269 19031 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:32.218927 18988 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.226832 18920 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.130048 18953 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.219406 18544 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.231056 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.231046 18998 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.220626 18699 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.251618 18920 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.237259 19040 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:32.247684 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:32.237918 18951 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.225078 18823 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.248262 18996 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:32.242637 19007 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.253870 18988 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:32.254417 19029 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:18:32.238951 19016 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.238566 18738 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.243321 19037 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.235539 18908 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:32.257769 17411 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:32.262770 19007 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.266896 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2292) took 1598 ms. Trace:
I20260709 13:18:32.267060 18699 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.267464 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 453) took 1158 ms. Trace:
I20260709 13:18:32.267613 18681 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:32.268093 16708 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.729s (SENT)
I20260709 13:18:32.271942 16587 rpcz_store.cc:276] 0709 13:18:30.668900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.672909 (+  4009us) service_pool.cc:224] Handling call
0709 13:18:30.680004 (+  7095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:30.702136 (+ 22132us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:30.702417 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.900270 (+197853us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:30.911786 (+ 11516us) log.cc:844] Serialized 26 byte log entry
0709 13:18:30.998388 (+ 86602us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:30.998441 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.998536 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:31.633203 (+634667us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:32.265266 (+632063us) raft_consensus.cc:1685] finished
0709 13:18:32.266653 (+  1387us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.266814 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3944,"prepare.queue_time_us":11078,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":3076,"raft.queue_time_us":4207,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":5670,"spinlock_wait_cycles":35419008,"thread_start_us":193258,"threads_started":1,"wal-append.queue_time_us":330883}
I20260709 13:18:32.274646 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:32.252635 19010 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.264024 19040 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:32.259277 18998 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.262614 18988 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.259729 18996 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:32.289294 16708 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.788s (SENT)
I20260709 13:18:32.277882 18953 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:32.290005 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:32.294066 18597 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.277539 18699 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:18:32.268473 19037 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.297519 18953 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:32.302826 16708 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.037s (SENT)
I20260709 13:18:32.293052 18745 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:32.303524 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.303884 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:32.272644 17414 rpcz_store.cc:276] 0709 13:18:31.108925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.109075 (+   150us) service_pool.cc:224] Handling call
0709 13:18:31.109471 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.126665 (+ 17194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.126719 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:31.127318 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.154231 (+ 26913us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:31.154288 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:31.154511 (+   223us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.263844 (+1109333us) raft_consensus.cc:1685] finished
0709 13:18:32.265219 (+  1375us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.267400 (+  2181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72619,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":26491,"threads_started":1,"wal-append.queue_time_us":964269}
I20260709 13:18:32.305054 19040 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.346762 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 909) took 2680 ms. Trace:
I20260709 13:18:32.349431 19010 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.353435 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:32.353857 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.357246 16738 rpcz_store.cc:276] 0709 13:18:29.666042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.863679 (+197637us) service_pool.cc:224] Handling call
0709 13:18:29.864043 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:29.864422 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:18:29.864454 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:29.871265 (+  6811us) log.cc:844] Serialized 26 byte log entry
0709 13:18:31.321889 (+1450624us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b1800457ae0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:18:32.093514 (+771625us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:32.093555 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.093623 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.325328 (+231705us) raft_consensus.cc:1685] finished
0709 13:18:32.325361 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.346664 (+ 21303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1578,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":20360,"spinlock_wait_cycles":8616320,"thread_start_us":771373,"threads_started":1,"wal-append.queue_time_us":2374318}
I20260709 13:18:32.366653 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.315181 18920 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.370999 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:32.371851 17416 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.395653 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.342036 18908 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:32.399394 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 137169us
I20260709 13:18:32.402805 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2234) took 1340 ms. Trace:
I20260709 13:18:32.403965 16742 rpcz_store.cc:276] 0709 13:18:31.061992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:31.148578 (+ 86586us) service_pool.cc:224] Handling call
0709 13:18:31.148838 (+   260us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:31.150760 (+  1922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.150805 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:31.151655 (+   850us) log.cc:844] Serialized 52 byte log entry
0709 13:18:32.095888 (+944233us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:32.355122 (+259234us) log.cc:844] Serialized 14 byte log entry
0709 13:18:32.402482 (+ 47360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.402559 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.402600 (+    41us) raft_consensus.cc:1685] finished
0709 13:18:32.402611 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.402749 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62624,"prepare.queue_time_us":253910,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":50,"raft.queue_time_us":51221,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":35210,"thread_start_us":1100772,"threads_started":2,"wal-append.queue_time_us":1013976}
W20260709 13:18:32.403798 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 100986us
I20260709 13:18:32.378921 19043 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:32.364208 18745 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.370601 18597 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.402776 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:32.462936 16742 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:18:32.466331 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:32.496582 16738 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.470479 16740 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:32.498574 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.370623 18681 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:32.407697 18961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.409s	user 0.017s	sys 0.020s
I20260709 13:18:32.460148 19010 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.473348 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:32.509292 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 106594us
W20260709 13:18:32.471149 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.509699 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.499439 16740 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:32.513981 18745 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.549567 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:32.554551 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:32.555509 16740 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.559386 19010 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.560169 19010 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.562894 16744 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:32.470160 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:32.569537 17411 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.549710 18932 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:32.515594 18961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.517s	user 0.017s	sys 0.021s
I20260709 13:18:32.407613 19041 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.160s
I20260709 13:18:32.471032 18738 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.575380 19056 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.549206 18597 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.587704 19056 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.589586 18597 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.590024 19056 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.567344 18681 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.577674 19016 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.579160 19060 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:32.593039 19060 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.593456 18823 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.471592 19044 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.604911 16739 raft_consensus.cc:2410] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:32.606549 17416 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:32.606565 16744 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:32.608856 16740 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
W20260709 13:18:32.583755 19057 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:18:32.620282 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.609330 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:32.629547 16738 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:32.632671 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.591867 18832 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.580262 18609 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:32.593186 18681 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.586143 18866 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.578969 19053 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.582146 19007 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.578006 18932 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.580652 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.657719 18755 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:32.597782 18951 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:32.610776 18992 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 2.172s	user 0.009s	sys 0.011s
I20260709 13:18:32.662032 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 149) took 2950 ms. Trace:
I20260709 13:18:32.593926 18988 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.585793 18934 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.619900 19011 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:32.655416 19053 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.665789 19011 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.667976 19053 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.668622 19011 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.634884 18823 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.662577 16596 rpcz_store.cc:276] 0709 13:18:29.711445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.757333 (+ 45888us) service_pool.cc:224] Handling call
0709 13:18:29.805517 (+ 48184us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:29.813232 (+  7715us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:29.813288 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:30.341491 (+528203us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b1800402580. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a
0709 13:18:30.342766 (+  1275us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b2c000014c8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:18:30.348057 (+  5291us) log.cc:844] Serialized 52 byte log entry
0709 13:18:30.401177 (+ 53120us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:30.433272 (+ 32095us) log.cc:844] Serialized 14 byte log entry
0709 13:18:30.433486 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:30.433557 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:30.577325 (+143768us) spinlock_profiling.cc:243] Waited 7.39 ms on lock 0x7b4800002958. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:18:30.580271 (+  2946us) spinlock_profiling.cc:243] Waited 3.67 ms on lock 0x7b1800402580. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db
0709 13:18:32.578498 (+1998227us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:32.661704 (+ 83206us) raft_consensus.cc:1685] finished
0709 13:18:32.661736 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:32.661943 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":172799,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":5325,"raft.run_cpu_time_us":1721,"raft.run_wall_time_us":235405,"spinlock_wait_cycles":115207552,"thread_start_us":77844,"threads_started":2,"wal-append.queue_time_us":90031}
I20260709 13:18:32.669958 18823 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.655406 18898 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.655706 19007 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:32.672130 19007 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:32.673386 19007 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.674211 18745 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.657255 18932 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:32.678478 16665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 191) took 3577 ms (client timeout 2032 ms). Trace:
W20260709 13:18:32.679163 16665 rpcz_store.cc:269] 0709 13:18:29.101538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.104973 (+  3435us) service_pool.cc:224] Handling call
0709 13:18:32.678138 (+3573165us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540012b6d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:32.678368 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3062649088}
I20260709 13:18:32.621672 18998 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:32.774665 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.000s	sys 0.016s
W20260709 13:18:32.775141 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.017s
W20260709 13:18:32.775894 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 596013us
I20260709 13:18:32.777253 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:32.773372 18823 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.638181 19037 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.508558 19043 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:32.797608 16590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 152) took 3067 ms (client timeout 2203 ms). Trace:
I20260709 13:18:32.795429 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:32.825661 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:32.793238 18681 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.801558 18934 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.611466 19044 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.778585 18832 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:32.839205 16590 rpcz_store.cc:269] 0709 13:18:29.729722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.096934 (+367212us) service_pool.cc:224] Handling call
0709 13:18:32.797329 (+2700395us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b540010cf54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:32.797510 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224718080}
I20260709 13:18:32.821226 18738 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.830772 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.855547 17413 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:32.851055 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55732.5us
I20260709 13:18:32.825222 19056 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:18:32.861837 16742 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:32.830133 18959 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:18:32.864980 18959 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:32.865391 18959 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:32.865851 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:32.867513 19070 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:32.868494 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:32.869087 18826 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.871042 16591 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:18:32.871193 16542 auto_rebalancer.cc:893] Removing replace marker failed: Leader has not yet committed an operation in its own term
W20260709 13:18:32.871655 16542 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 4951b0fadf5b43e49a71f0bb4fb7a85e already in progress
W20260709 13:18:32.871927 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 4951b0fadf5b43e49a71f0bb4fb7a85e already in progress
I20260709 13:18:32.873224 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.842187 19064 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:32.875907 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:32.876857 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.877223 18826 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.877609 18826 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.878032 18826 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.879006 19043 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.880223 19065 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.880587 19044 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:32.880679 16741 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:18:32.869084 16594 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:32.843570 18681 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.854795 18832 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.885075 18832 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.885197 18908 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:32.877686 16744 raft_consensus.cc:2410] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:32.853137 18738 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:32.887776 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.888360 16671 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:32.890324 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.891058 16665 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:32.892490 16740 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:32.852965 18934 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.893550 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.894114 16671 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:32.895948 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.896718 16738 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:32.898110 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.899313 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.899909 16744 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:32.900758 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.901662 16743 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:32.904285 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.905211 16743 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:32.900806 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.906368 16739 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:32.907656 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:32.907867 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.909360 16594 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:32.909504 16741 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:18:32.885098 18681 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.908577 16743 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:32.891885 18738 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.912998 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:32.913663 16740 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:32.890219 18908 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:32.915503 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:32.923507 17416 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:32.880765 19065 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.928710 18934 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:32.929203 19065 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:32.866346 18959 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:32.913334 18866 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:18:32.939738 18959 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:32.901224 19070 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:32.940707 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.941445 17413 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:32.923832 18998 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:32.942031 18998 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:32.942440 18998 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:32.943699 19016 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.944193 19016 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.947708 18908 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:32.948452 17374 connection.cc:256] RPC call timeout handler was delayed by -1.4423758559998987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:18:32.959754 16708 leader_election.cc:336] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.813s (SENT)
I20260709 13:18:32.974172 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2228) took 2077 ms. Trace:
I20260709 13:18:32.974699 16736 rpcz_store.cc:276] 0709 13:18:30.898659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.902767 (+  4108us) service_pool.cc:224] Handling call
0709 13:18:30.934764 (+ 31997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:31.246173 (+311409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.973860 (+1727687us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400049a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:32.974091 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714502016}
I20260709 13:18:32.941164 19070 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:32.982288 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:32.983289 17416 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
W20260709 13:18:32.988224 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:32.989012 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:32.989543 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.990403 16670 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:32.991569 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 80030.7us
I20260709 13:18:32.992055 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:32.952721 18755 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:32.998658 17419 raft_consensus.cc:2410] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:18:33.003007 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53995.6us
I20260709 13:18:33.006283 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.007401 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.008257 16668 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:33.009038 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.006533 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.009862 16664 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.010246 16669 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:33.010627 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:33.013692 17374 connection.cc:256] RPC call timeout handler was delayed by -1.0035958220000794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:18:33.014245 17374 connection.cc:256] RPC call timeout handler was delayed by -1.0034041840001464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24580000000000002s for a response
I20260709 13:18:33.015048 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.016667 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.984439 19056 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:32.950892 18826 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:32.982905 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.019608 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.029983 16663 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:18:33.021509 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.030666 16670 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
W20260709 13:18:33.025024 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 475547us
I20260709 13:18:32.985633 19070 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:18:33.027870 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 120267us
I20260709 13:18:32.975829 18916 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:18:32.985507 17411 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:32.996414 18547 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:33.044230 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.044751 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.012588 19065 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.041302 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:32.979147 18934 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:18:33.040576 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:32.999377 19033 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.051267 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.059130 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.065512 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:33.035441 19046 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:33.068835 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.016096 18755 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.059448 18920 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:33.061651 19065 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.052775 16665 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.073925 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.095364 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.072451 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.098510 16668 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:33.027307 19056 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:33.100606 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49422.1us
I20260709 13:18:33.106588 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.071049 19046 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:33.102195 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.111146 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.102854 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.112202 16670 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.108709 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:33.107302 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.786s (SENT)
I20260709 13:18:33.113157 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:33.112205 16663 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.115805 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.093393 18920 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.079954 19007 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.074802 19074 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.130363 19074 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:33.130645 16737 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:33.118477 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.055107 18547 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.135309 18547 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:33.096552 17410 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:33.097098 18953 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:33.139878 16663 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:33.142113 18866 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:33.143290 18866 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:33.143658 18866 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:33.145949 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.145641 16493 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:33.146653 16663 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:33.128484 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.129289 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.131868 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.106280 18832 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.148520 16665 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:33.133208 18755 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:33.147567 16493 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:18:33.130113 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84126.2us
I20260709 13:18:33.150758 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.144555 18866 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.115741 19053 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:33.148684 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:33.152695 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.097013 19069 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.115613 19040 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.120271 19067 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.068043 19033 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.127148 18898 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:33.123359 18609 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.127390 18920 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:33.132688 19071 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:18:33.165334 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.170624 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.114696 19046 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.162705 18866 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:33.172509 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.107563 18908 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:18:33.181663 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.162997 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.163431 18823 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:33.155401 18953 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:33.159755 19044 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.197592 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.179876 19016 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:33.199195 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.225845 16665 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:33.201181 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.226851 17415 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:33.164523 19067 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:33.179709 18932 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.001s
I20260709 13:18:33.240267 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.240864 18932 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:33.243449 18932 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:33.175827 18699 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:33.246030 19021 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.155490 18917 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:18:33.126991 19070 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.159857 18915 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:18:33.247546 18917 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.247951 18917 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:33.248143 18915 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:33.248435 18996 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:33.223064 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.247546 19070 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.248869 18917 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.249689 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.250168 18917 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.251639 18960 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:33.252197 18960 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:33.253079 18960 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:33.254391 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.173179 18918 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.223237 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.255008 18434 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:33.255697 18434 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:33.256254 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.256913 16663 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:33.224081 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 116795us
I20260709 13:18:33.257458 18959 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:33.257845 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.258509 16663 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
W20260709 13:18:33.259021 17374 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.032s (SENT)
I20260709 13:18:33.157205 18832 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:33.260270 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.260865 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.261821 18832 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:33.263085 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.263682 17415 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.265165 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2299) took 2586 ms. Trace:
W20260709 13:18:33.231292 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.260865 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.234969 19088 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:33.239356 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.267210 19088 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:33.267450 16665 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.268167 19088 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.269057 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.269886 16665 raft_consensus.cc:2376] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:33.224892 19044 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:33.271071 19044 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.246601 19021 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.272229 19021 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.271044 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.272768 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.273037 16665 raft_consensus.cc:2376] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:33.273983 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.274497 16665 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.275326 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:33.275768 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.277141 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.278407 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:33.279002 19060 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:33.280270 19016 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:33.248598 18915 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:33.280853 19016 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.281237 19016 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.281419 18915 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.282694 18915 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.256516 18434 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.237828 19010 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:33.277176 17415 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: Advancing to term 5
I20260709 13:18:33.265587 16589 rpcz_store.cc:276] 0709 13:18:30.679104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:30.753479 (+ 74375us) service_pool.cc:224] Handling call
0709 13:18:31.376815 (+623336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:31.377245 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:31.377277 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:32.108686 (+731409us) log.cc:844] Serialized 26 byte log entry
0709 13:18:32.350426 (+241740us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:32.350481 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:32.350591 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:32.828646 (+478055us) raft_consensus.cc:1685] finished
0709 13:18:32.828680 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.264917 (+436237us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b5400190fd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:33.265100 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5114,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":1432435968,"thread_start_us":157717,"threads_started":1,"wal-append.queue_time_us":185577}
I20260709 13:18:33.280262 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.275789 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.206354 18547 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:33.212906 19043 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.297967 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:33.294521 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 123163us
I20260709 13:18:33.295800 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.299539 16663 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.300849 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.294142 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:33.304097 16665 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:33.304713 16594 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:18:33.305054 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.303555 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.306245 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.255086 18918 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.307087 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.233076 18823 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:33.312461 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.315444 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.307435 16665 raft_consensus.cc:2410] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:33.309743 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.184672 19084 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.109s
I20260709 13:18:33.285225 19067 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.304028 18547 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.303530 18832 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:33.316177 16663 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:33.328634 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.329569 16663 raft_consensus.cc:2410] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:33.330415 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.318514 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.322072 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.332460 16665 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:33.333608 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:33.316177 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 60567.2us
I20260709 13:18:33.323225 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.324838 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.334972 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.344590 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.313530 18918 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:33.348893 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51039us
I20260709 13:18:33.357358 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:33.362375 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.363341 16594 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:33.363466 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.364262 16742 raft_consensus.cc:2393] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 3: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:33.364562 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.365260 16591 raft_consensus.cc:2376] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4. Re-sending same reply.
I20260709 13:18:33.368024 16742 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:33.370069 16665 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:33.372900 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:33.375605 16738 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:18:33.377023 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.326534 19070 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.382196 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.322364 18959 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.377701 16593 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:33.325627 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:33.221068 19074 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.308377 19043 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.332111 18547 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:33.359532 18908 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:33.267623 18953 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.387660 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.324479 18738 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.393294 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 271) took 1267 ms. Trace:
I20260709 13:18:33.378736 18826 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:33.413795 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.438607 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.323117 19087 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:33.359472 19065 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:33.419669 16745 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:18:33.420367 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 126138us
I20260709 13:18:33.420945 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.425292 16665 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.433261 16737 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:33.433955 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:33.435055 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:33.448870 16738 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:33.449154 16739 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:33.424777 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:33.441753 16664 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.441923 16662 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:33.387041 19070 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.448246 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.457345 16662 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:18:33.457465 16745 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:33.458106 16662 pending_rounds.cc:85] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 1
I20260709 13:18:33.458595 16662 pending_rounds.cc:107] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:18:33.454986 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:33.458956 16662 raft_consensus.cc:2889] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:33.460986 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.461361 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.462182 16745 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:33.463294 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:33.463912 16744 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.461675 16737 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:33.377624 19089 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.095s
I20260709 13:18:33.442531 18972 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:33.442664 18961 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:33.435249 19056 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:33.483563 16662 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.481022 16739 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:18:33.493534 16662 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:33.490700 16561 connection.cc:256] RPC call timeout handler was delayed by -1.708307017999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1597s for a response
W20260709 13:18:33.494273 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4194070179999017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
I20260709 13:18:33.496533 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.496569 16738 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:33.497574 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.498063 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.498431 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.498754 16738 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:33.504329 16738 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:33.506292 16738 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:33.508424 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:33.509024 16744 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:33.510493 16744 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.512642 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.513604 16594 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:33.514417 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.515980 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.517336 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.518069 16594 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.519289 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.453382 18916 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:33.520874 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:33.521757 16744 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:33.472033 18998 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.001s
I20260709 13:18:33.460506 19070 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.525983 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.526590 16740 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:33.541333 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.543278 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:33.550367 16745 raft_consensus.cc:2410] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:18:33.543430 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.551102 16596 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:33.543298 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.544813 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.461189 18908 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:33.559501 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 104664us
I20260709 13:18:33.504127 18961 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.494781 18934 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.546373 16662 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.547887 16587 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:33.511945 19053 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.545729 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.552971 16744 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:18:33.554656 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 197363us
I20260709 13:18:33.525890 19010 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.492049 18972 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:33.561762 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.472316 18996 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:33.585300 16662 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:33.566897 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.568290 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.526472 18804 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.557315 18738 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:33.544076 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.594117 16593 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.499830 19044 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.559536 19089 consensus_queue.cc:1243] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 5.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:18:33.591064 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.587452 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:33.595222 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 122726us
I20260709 13:18:33.598035 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.600662 16587 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:33.601716 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.462170 18823 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.565059 18961 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:33.602345 16587 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:33.606766 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.607213 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:33.607558 16587 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.608695 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:33.547617 18998 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.559227 18916 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.589426 18972 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:33.585807 18996 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:33.622207 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.631868 16591 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:33.612962 16587 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:18:33.623003 16736 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:33.623850 16663 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 2. Current term is 3. Ops: []
I20260709 13:18:33.625023 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.633241 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:33.633026 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 260031us
I20260709 13:18:33.579346 19053 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.572069 18934 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:33.599258 18804 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.604640 19088 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:18:34.001425 19043 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:34.358701 19067 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.009s	user 0.013s	sys 0.000s
I20260709 13:18:34.377250 18898 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:34.361363 18961 consensus_meta.cc:287] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.737s	user 0.010s	sys 0.001s
W20260709 13:18:33.870365 19068 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.007s	sys 0.003s
W20260709 13:18:33.700315 19102 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:18:33.863822 18920 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.680s	user 0.004s	sys 0.005s
I20260709 13:18:33.674383 18951 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:34.002905 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:34.445531 19089 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:33.690889 17415 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
W20260709 13:18:33.671408 19007 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.001s
W20260709 13:18:34.008379 18547 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.000s
I20260709 13:18:33.183387 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:34.542398 17415 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 3
I20260709 13:18:34.543290 17415 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 4.4
W20260709 13:18:33.861063 19107 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.299s	user 0.001s	sys 0.001s
I20260709 13:18:34.543526 17415 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:34.450407 19064 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:33.635057 18996 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:33.703819 19092 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 13:18:34.551764 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:33.966430 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.656242 18916 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:33.854918 19033 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:33.961009 18932 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:34.289119 16492 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 233) took 17748 ms (client timeout 15000 ms). Trace:
I20260709 13:18:33.919291 18972 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:33.965152 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:33.674975 16590 raft_consensus.cc:2410] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:34.288014 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:33.871074 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:33.681979 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 47014.9us
W20260709 13:18:34.461058 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 830448us
I20260709 13:18:34.560696 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:34.562050 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:33.911139 16593 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:33.653743 16670 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:33.862212 17419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 404) took 4661 ms (client timeout 3058 ms). Trace:
I20260709 13:18:33.634411 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:33.659678 16596 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
W20260709 13:18:34.563004 17419 rpcz_store.cc:269] 0709 13:18:29.200469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.200591 (+   122us) service_pool.cc:224] Handling call
0709 13:18:33.861940 (+4661349us) spinlock_profiling.cc:243] Waited 3.76 s on lock 0x7b5400484254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:33.862118 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8640809856}
I20260709 13:18:33.436724 17409 rpcz_store.cc:276] 0709 13:18:32.125451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.135083 (+  9632us) service_pool.cc:224] Handling call
0709 13:18:33.281853 (+1146770us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:33.282098 (+   245us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:33.282520 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.282557 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:33.283126 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:18:33.348000 (+ 64874us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:33.348060 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.348140 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:33.393005 (+ 44865us) raft_consensus.cc:1685] finished
0709 13:18:33.393038 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:33.393234 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7423,"raft.queue_time_us":1511,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":8239,"spinlock_wait_cycles":17932032,"thread_start_us":64432,"threads_started":1,"wal-append.queue_time_us":105281}
I20260709 13:18:33.750902 16493 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:34.566284 17420 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:18:34.497874 18920 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.483525 18898 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.566138 16493 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:33.672905 18918 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:33.639942 16663 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:34.579515 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1083) took 1010 ms. Trace:
I20260709 13:18:33.965852 17416 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:18:33.960564 19046 consensus_meta.cc:287] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.762s	user 0.007s	sys 0.003s
I20260709 13:18:34.488554 16595 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:33.732060 19114 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:18:33.703187 19105 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:18:33.667353 19104 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:18:33.955054 19016 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.001s
W20260709 13:18:33.694372 19103 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 13:18:34.499140 16594 raft_consensus.cc:2410] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:33.957191 18434 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:18:34.063478 18866 consensus_meta.cc:287] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.002s
I20260709 13:18:33.995743 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:33.673955 19094 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:18:33.667629 17414 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:33.757648 19090 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.338s	user 0.003s	sys 0.000s
W20260709 13:18:33.961011 18915 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.003s
I20260709 13:18:34.555083 19064 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.668092 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:34.477979 19074 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:34.497943 18951 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:34.544288 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 418) took 1269 ms. Trace:
I20260709 13:18:34.546080 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:33.657529 18998 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.542680 19007 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:33.681177 19021 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.542917 18547 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.706537 18972 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.711236 16662 raft_consensus.cc:2368] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:34.714434 16739 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
W20260709 13:18:34.550318 19118 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:18:34.734624 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2253) took 1478 ms. Trace:
I20260709 13:18:34.554276 16665 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:18:33.704259 19070 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:33.997032 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:33.710300 19056 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.840s
W20260709 13:18:34.556341 16559 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.031s	sys 0.040s
W20260709 13:18:34.742403 16559 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.032s	sys 0.040s
W20260709 13:18:34.742798 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 782318us
I20260709 13:18:34.470836 19067 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.562793 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1760) took 1067 ms. Trace:
I20260709 13:18:34.745186 16593 rpcz_store.cc:276] 0709 13:18:33.495458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.495638 (+   180us) service_pool.cc:224] Handling call
0709 13:18:34.562739 (+1067101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.566174 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.151191 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.545287 19044 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:34.305678 16708 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.530s (SENT)
I20260709 13:18:34.596143 16741 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:34.746708 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1072) took 1683 ms. Trace:
I20260709 13:18:34.629164 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:34.747563 17419 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:34.578511 16493 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 4, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:18:34.746758 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:34.748976 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 443308us
I20260709 13:18:34.748668 16493 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:18:34.579849 16663 rpcz_store.cc:276] 0709 13:18:33.569233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.633986 (+ 64753us) service_pool.cc:224] Handling call
0709 13:18:33.634251 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.579242 (+944991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.579459 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.635221 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:34.635815 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 54072.9us
I20260709 13:18:33.708227 18908 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.487957 18961 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.753108 16663 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:18:34.754336 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1787) took 1155 ms. Trace:
I20260709 13:18:34.754777 18961 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:34.645114 16738 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:34.756124 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:34.756737 16738 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:34.757081 19067 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:34.545432 19089 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.759858 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1766) took 1262 ms. Trace:
I20260709 13:18:34.760416 19089 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.652990 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.766250 16591 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:34.766191 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1749) took 1191 ms. Trace:
I20260709 13:18:34.655719 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:34.573727 19085 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:34.766940 16662 rpcz_store.cc:276] 0709 13:18:33.575238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.608180 (+ 32942us) service_pool.cc:224] Handling call
0709 13:18:34.765948 (+1157768us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540009e054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:34.766122 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477302016}
I20260709 13:18:34.767740 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1775) took 1267 ms. Trace:
I20260709 13:18:34.768343 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:34.768257 16594 rpcz_store.cc:276] 0709 13:18:33.500184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.545224 (+ 45040us) service_pool.cc:224] Handling call
0709 13:18:34.767503 (+1222279us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b54002378d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:34.767666 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134684032}
I20260709 13:18:34.769722 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.650529 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:34.660641 16588 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:34.576885 19116 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:34.655207 18434 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:34.776500 18434 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.777520 18434 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.777364 18951 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.629391 19099 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.464978 18544 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.630985 19043 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:34.779107 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2317) took 1047 ms. Trace:
I20260709 13:18:34.769219 16662 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:34.779520 19043 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.780783 19021 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:34.781657 19043 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:34.490057 19068 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.389s	user 0.007s	sys 0.020s
I20260709 13:18:34.671571 18804 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.672665 19053 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.782461 18934 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.574055 18932 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:34.556169 16492 rpcz_store.cc:269] 0709 13:18:16.540685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:16.541045 (+   360us) service_pool.cc:224] Handling call
0709 13:18:34.288831 (+17747786us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:27.726130 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:27.731465 (+  5335us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:27.731546 (+    81us) write_op.cc:435] Acquired schema lock
0709 13:18:27.731591 (+    45us) tablet.cc:662] Decoding operations
0709 13:18:27.732827 (+  1236us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:27.732930 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:27.732971 (+    41us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:18:27.908865 (+175894us) tablet.cc:701] Row locks acquired
0709 13:18:27.908918 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:18:27.909122 (+   204us) write_op.cc:270] Start()
0709 13:18:27.909305 (+   183us) write_op.cc:276] Timestamp: P: 1783603107909043 usec, L: 0
0709 13:18:27.909345 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:18:27.910112 (+   767us) log.cc:844] Serialized 3643 byte log entry
0709 13:18:29.091136 (+1181024us) op_driver.cc:464] REPLICATION: finished
0709 13:18:29.334891 (+243755us) write_op.cc:301] APPLY: starting
0709 13:18:29.341662 (+  6771us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:29.367329 (+ 25667us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:29.367474 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:29.378421 (+ 10947us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:29.378741 (+   320us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:18:29.378797 (+    56us) write_op.cc:312] APPLY: finished
0709 13:18:29.379179 (+   382us) log.cc:844] Serialized 71 byte log entry
0709 13:18:29.379367 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:29.379486 (+   119us) write_op.cc:454] Released schema lock
0709 13:18:29.379546 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:29.379978 (+   432us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:29.380005 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4182690,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":225981,"apply.run_cpu_time_us":4324,"apply.run_wall_time_us":55721,"num_ops":9,"prepare.queue_time_us":204134,"prepare.run_cpu_time_us":46427,"prepare.run_wall_time_us":508503,"raft.queue_time_us":474897,"raft.run_cpu_time_us":36978,"raft.run_wall_time_us":170074,"replication_time_us":1181541,"thread_start_us":771493,"threads_started":4,"wal-append.queue_time_us":499210,"wal-append.run_cpu_time_us":20373,"wal-append.run_wall_time_us":1973786}]]}
I20260709 13:18:34.676501 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:34.789251 17416 raft_consensus.cc:2410] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:18:34.637357 19046 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.684947 18898 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:34.690264 19064 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.647516 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:34.656823 19123 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.648034 19010 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:18:34.692637 16740 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:34.773273 18597 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:34.693205 17415 rpcz_store.cc:276] 0709 13:18:33.275119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.276740 (+  1621us) service_pool.cc:224] Handling call
0709 13:18:33.277052 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.544042 (+1266990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.544234 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.704826 16592 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:34.666507 17411 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:34.727149 18918 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:34.727837 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:34.740556 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:34.743788 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:34.801478 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1080) took 1675 ms. Trace:
I20260709 13:18:34.752969 18998 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:34.647593 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 168340us
W20260709 13:18:34.801725 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61299.4us
I20260709 13:18:34.802721 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:34.761178 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1780) took 1213 ms. Trace:
I20260709 13:18:34.803653 16590 rpcz_store.cc:276] 0709 13:18:33.547391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.552503 (+  5112us) service_pool.cc:224] Handling call
0709 13:18:34.760991 (+1208488us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b5400157f54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:34.761119 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":202096384}
I20260709 13:18:34.660400 19016 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.760282 16595 rpcz_store.cc:276] 0709 13:18:33.497337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.497450 (+   113us) service_pool.cc:224] Handling call
0709 13:18:34.759679 (+1262229us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54001fc2d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:34.759801 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2160554752}
I20260709 13:18:34.801914 16664 rpcz_store.cc:276] 0709 13:18:33.126153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.572753 (+446600us) service_pool.cc:224] Handling call
0709 13:18:33.581857 (+  9104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:33.598787 (+ 16930us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.598819 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:33.609790 (+ 10971us) log.cc:844] Serialized 26 byte log entry
0709 13:18:33.915877 (+306087us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:33.915941 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.916060 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:34.801257 (+885197us) raft_consensus.cc:1685] finished
0709 13:18:34.801279 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.801419 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16451,"raft.queue_time_us":22604,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12372,"thread_start_us":198571,"threads_started":1,"wal-append.queue_time_us":816973}
I20260709 13:18:34.806962 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:34.807634 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.808276 16745 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:34.808367 16595 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:34.808967 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2313) took 1096 ms. Trace:
I20260709 13:18:34.747251 16559 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:34.809262 16595 rpcz_store.cc:276] 0709 13:18:33.730749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.807382 (+1076633us) service_pool.cc:224] Handling call
0709 13:18:34.808909 (+  1527us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.809506 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:34.746492 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.754733 16596 rpcz_store.cc:276] 0709 13:18:33.598514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.601413 (+  2899us) service_pool.cc:224] Handling call
0709 13:18:34.754164 (+1152751us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b54001919d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:34.754277 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65028864}
I20260709 13:18:34.747129 16670 rpcz_store.cc:276] 0709 13:18:33.063164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.510083 (+446919us) service_pool.cc:224] Handling call
0709 13:18:33.510329 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.562787 (+1052458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.746492 (+183705us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001f6b54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:34.746647 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325114240}
I20260709 13:18:34.810784 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.735205 16743 rpcz_store.cc:276] 0709 13:18:33.255893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.260941 (+  5048us) service_pool.cc:224] Handling call
0709 13:18:33.261256 (+   315us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:33.261640 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.261666 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:33.262236 (+   570us) log.cc:844] Serialized 52 byte log entry
0709 13:18:33.346172 (+ 83936us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:33.387737 (+ 41565us) log.cc:844] Serialized 14 byte log entry
0709 13:18:33.388227 (+   490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.388320 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:33.665756 (+277436us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b18004589e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7557 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:34.734194 (+1068438us) raft_consensus.cc:1685] finished
0709 13:18:34.734341 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.734542 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":114505,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1106,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":199,"spinlock_wait_cycles":10815360,"thread_start_us":124479,"threads_started":2,"wal-append.queue_time_us":250228}
I20260709 13:18:34.763967 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:34.761176 16671 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:34.765233 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1718) took 1777 ms. Trace:
I20260709 13:18:34.647717 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:34.770505 16594 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:34.764870 19007 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:34.780035 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2316) took 1048 ms. Trace:
I20260709 13:18:34.673918 18920 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:34.663311 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:34.824783 18932 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:34.799396 18597 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:34.826228 18597 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.827461 18597 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:34.829144 19070 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:34.790978 18996 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:34.830047 16592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1763) took 1333 ms (client timeout 1530 ms). Trace:
W20260709 13:18:34.830425 16592 rpcz_store.cc:269] 0709 13:18:33.496466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.497345 (+   879us) service_pool.cc:224] Handling call
0709 13:18:34.829887 (+1332542us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400051c54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:34.829987 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2745657088}
I20260709 13:18:34.831233 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2310) took 1171 ms. Trace:
I20260709 13:18:34.779137 19099 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.831415 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.832134 17419 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:34.832162 16592 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:34.786813 16493 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3, now has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } } pending_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:34.814925 18972 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:18:34.771477 16737 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:34.786106 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2319) took 1053 ms. Trace:
I20260709 13:18:34.779562 16587 rpcz_store.cc:276] 0709 13:18:33.732193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.565932 (+833739us) service_pool.cc:224] Handling call
0709 13:18:34.779047 (+213115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.733933 18915 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:34.795781 16560 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2323) took 1061 ms. Trace:
I20260709 13:18:34.796181 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:34.941279 16560 rpcz_store.cc:276] 0709 13:18:33.734161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.795718 (+1061557us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:18:34.695942 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.270s (SENT) [suppressed 52 similar messages]
I20260709 13:18:34.941540 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.941601 17416 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:34.797642 16738 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:34.779549 18547 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:34.752337 16708 leader_election.cc:336] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.764s (SENT)
I20260709 13:18:34.972246 16493 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:34.972837 18915 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:34.973340 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:34.974236 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 221913us
I20260709 13:18:34.973966 16493 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:34.804864 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:34.975677 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2318) took 1243 ms. Trace:
I20260709 13:18:34.976109 16590 rpcz_store.cc:276] 0709 13:18:33.732608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.804511 (+1071903us) service_pool.cc:224] Handling call
0709 13:18:34.975611 (+171100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:34.976485 16708 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.217s (SENT)
I20260709 13:18:34.977027 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:34.977221 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:34.831337 18996 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:34.977972 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1793) took 1341 ms. Trace:
I20260709 13:18:34.805426 17415 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: Advancing to term 5
I20260709 13:18:34.979064 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:34.783177 19067 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:34.763453 16742 raft_consensus.cc:2368] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5.
I20260709 13:18:34.810626 16593 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:34.812603 16743 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:18:34.809962 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62776.6us
I20260709 13:18:34.814915 16589 rpcz_store.cc:276] 0709 13:18:33.731845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.150784 (+418939us) service_pool.cc:224] Handling call
0709 13:18:34.779976 (+629192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.813042 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2321) took 1079 ms. Trace:
I20260709 13:18:34.817750 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:34.814218 18934 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:34.794572 19043 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:34.798781 19123 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.812870 18898 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:18:34.820500 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 412) took 1909 ms. Trace:
I20260709 13:18:34.820943 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:34.822619 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:34.832729 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1791) took 1206 ms. Trace:
I20260709 13:18:34.814265 17412 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:35.019513 17416 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:34.813282 18998 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:34.812405 16667 rpcz_store.cc:276] 0709 13:18:32.987633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.987759 (+   126us) service_pool.cc:224] Handling call
0709 13:18:33.014728 (+ 26969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:33.082145 (+ 67417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:33.082200 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:33.098942 (+ 16742us) log.cc:844] Serialized 26 byte log entry
0709 13:18:33.229655 (+130713us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:33.229721 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:33.229806 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:33.756855 (+527049us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:34.737217 (+980362us) raft_consensus.cc:1685] finished
0709 13:18:34.737249 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:34.765157 (+ 27908us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1455,"raft.queue_time_us":61161,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":14876,"spinlock_wait_cycles":13733248,"thread_start_us":128525,"threads_started":2,"wal-append.queue_time_us":196842}
I20260709 13:18:34.940606 16737 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 13:18:35.020872 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:34.834048 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:35.021514 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2309) took 1368 ms. Trace:
I20260709 13:18:35.016614 18544 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.021915 16744 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:35.021943 16596 rpcz_store.cc:276] 0709 13:18:33.652768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.811251 (+1158483us) service_pool.cc:224] Handling call
0709 13:18:35.021458 (+210207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.831589 16595 rpcz_store.cc:276] 0709 13:18:33.660273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.810206 (+1149933us) service_pool.cc:224] Handling call
0709 13:18:34.831171 (+ 20965us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.023376 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:34.960938 16560 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:38576 dropped due to backpressure: service queue is full with 50 items
I20260709 13:18:34.969884 16736 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:18:34.971396 16745 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:34.960865 16587 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:35.024598 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2307) took 1400 ms. Trace:
I20260709 13:18:34.984608 19084 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:35.025058 16587 rpcz_store.cc:276] 0709 13:18:33.623976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.878305 (+1254329us) service_pool.cc:224] Handling call
0709 13:18:35.024551 (+146246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.983201 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.021693 18996 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:34.977519 19126 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:34.813427 19083 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:34.984769 16738 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:18:34.985747 16593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2322) took 1252 ms (client timeout 1656 ms). Trace:
I20260709 13:18:35.037766 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:35.037712 16593 rpcz_store.cc:269] 0709 13:18:33.733827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.746207 (+1012380us) service_pool.cc:224] Handling call
0709 13:18:34.985692 (+239485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.001593 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:35.038779 17415 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:18:34.986760 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.040197 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2315) took 1308 ms. Trace:
I20260709 13:18:35.040596 16589 rpcz_store.cc:276] 0709 13:18:33.731496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.986530 (+1255034us) service_pool.cc:224] Handling call
0709 13:18:35.040132 (+ 53602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.986922 16594 rpcz_store.cc:276] 0709 13:18:33.733536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.769423 (+1035887us) service_pool.cc:224] Handling call
0709 13:18:34.812975 (+ 43552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.989630 16743 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:34.987470 17420 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:35.004192 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:34.813064 19089 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.017035 16740 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:35.016938 16592 rpcz_store.cc:276] 0709 13:18:33.626174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.831191 (+1205017us) service_pool.cc:224] Handling call
0709 13:18:34.832671 (+  1480us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.877054 16591 rpcz_store.cc:276] 0709 13:18:33.732918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.634733 (+901815us) service_pool.cc:224] Handling call
0709 13:18:34.786024 (+151291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.024921 18544 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.016222 18681 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:35.026569 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:34.975690 16493 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:34.978349 16590 rpcz_store.cc:276] 0709 13:18:33.636111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.976956 (+1340845us) service_pool.cc:224] Handling call
0709 13:18:34.977921 (+   965us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:34.995080 19010 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:34.995309 18915 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.016567 17414 rpcz_store.cc:276] 0709 13:18:32.911263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:32.911821 (+   558us) service_pool.cc:224] Handling call
0709 13:18:32.930040 (+ 18219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.667748 (+1737708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:34.820241 (+152493us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54005d25d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:34.820432 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676267136}
I20260709 13:18:35.006107 18934 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.026775 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.013907 18597 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.000051 19007 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.035830 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2312) took 1323 ms. Trace:
I20260709 13:18:35.035470 19044 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:35.025316 19084 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.048650 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:35.100663 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52073.4us
I20260709 13:18:35.049408 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1778) took 1522 ms. Trace:
I20260709 13:18:35.101786 16596 rpcz_store.cc:276] 0709 13:18:33.527438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.023025 (+1495587us) service_pool.cc:224] Handling call
0709 13:18:35.049348 (+ 26323us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:35.049825 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.052774 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:35.055737 18918 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:35.006284 19067 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.039573 17415 pending_rounds.cc:85] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 3
I20260709 13:18:35.098397 18544 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:35.076002 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.033882 17409 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:35.103834 19021 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:35.104216 18742 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:35.104620 18742 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.104982 18742 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:35.098558 19007 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:35.105716 19007 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.107839 19046 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.109417 18998 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.019917 19043 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.110445 19067 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.112051 19043 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:35.097433 16595 rpcz_store.cc:276] 0709 13:18:33.712250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.023136 (+1310886us) service_pool.cc:224] Handling call
0709 13:18:35.035760 (+ 12624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.033974 18951 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:35.096498 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:35.103425 17415 pending_rounds.cc:107] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:18:35.056346 18934 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.115486 17415 raft_consensus.cc:2889] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.116739 18951 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:35.117427 18951 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:35.118920 19122 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:35.099000 19126 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:35.119485 19122 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:35.104256 19021 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:35.120044 19021 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:35.118494 18951 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.120348 19122 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.053503 16493 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:35.033779 18547 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:35.103259 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50602.4us
I20260709 13:18:35.111029 19044 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:35.126487 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.056388 18823 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:35.098026 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 71306.2us
W20260709 13:18:35.091136 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.037153 19083 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.047580 19089 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.124945 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.052582 19129 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:35.129340 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.159719 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:35.120124 19074 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:18:35.029078 19064 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:35.047758 18996 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:35.155612 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 459661us
I20260709 13:18:35.053808 19056 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:35.055733 17413 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:35.123447 19084 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.160426 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.159080 19083 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.024004 16560 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:38576 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1794)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1799)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1802)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1790)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1803)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1807)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2339)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1800)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1795)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1805)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2308)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2340)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1801)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2336)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2327)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1796)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1797)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1798)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2338)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1806)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1773)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2311)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2314)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2320)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2324)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2325)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2326)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1804)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2328)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2329)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2330)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2331)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2332)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2333)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2334)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2335)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2337)
I20260709 13:18:35.156774 19043 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.183996 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:35.101480 18597 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.158319 19089 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.185966 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:35.186854 17420 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 3 LEARNER]: Advancing to term 5
I20260709 13:18:35.170603 18960 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.191536 16740 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:35.100769 19053 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:18:35.175122 18823 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:35.178123 19010 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:35.119493 19126 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.056895 18681 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.183884 18998 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:35.158124 18804 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:18:35.189834 19067 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:18:35.193487 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.189868 19043 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.156020 18959 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:18:35.132525 18547 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.196710 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.196259 19010 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.168838 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:35.132061 18932 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:18:35.160249 18915 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:18:35.201368 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 256) took 1199 ms. Trace:
I20260709 13:18:35.189659 18826 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:35.177928 19068 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:18:35.179593 19064 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.195864 18960 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.196978 17416 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: Advancing to term 6
I20260709 13:18:35.209748 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 942) took 1719 ms. Trace:
I20260709 13:18:35.213418 19070 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:35.206768 19043 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.217386 19007 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.219012 19126 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.224424 16739 rpcz_store.cc:276] 0709 13:18:34.002268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.002484 (+   216us) service_pool.cc:224] Handling call
0709 13:18:35.201202 (+1198718us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400427554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:35.201305 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496822016}
I20260709 13:18:35.224792 17420 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:35.195254 17411 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
W20260709 13:18:35.229259 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64696.7us
I20260709 13:18:35.230139 16741 rpcz_store.cc:276] 0709 13:18:33.490592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.512060 (+ 21468us) service_pool.cc:224] Handling call
0709 13:18:33.548779 (+ 36719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:34.746909 (+1198130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.209500 (+462591us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003d3954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:35.209670 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408059648}
I20260709 13:18:35.205098 18998 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.233050 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.206462 18920 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:35.210421 18804 consensus_queue.cc:1243] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:35.226011 19010 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:35.264616 17416 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:35.226749 18826 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.272259 16743 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:35.272923 17376 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.231595 18960 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.273612 17420 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:35.245254 19085 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:35.242347 19133 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:35.231519 19070 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:35.289794 16634 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:35.274735 16741 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:35.224540 18996 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.249143 18920 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.274040 18826 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:35.266422 17411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 468) took 1292 ms (client timeout 1657 ms). Trace:
I20260709 13:18:35.210098 19044 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.245447 18959 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:18:35.271184 17414 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:35.276059 18961 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:35.248852 18544 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:18:35.290401 19046 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:35.271950 19074 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:35.296954 18681 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:35.289439 19133 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:35.282277 18932 consensus_queue.cc:1243] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:35.294487 18920 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:35.295819 19085 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.303856 19068 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:35.343181 19068 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.345773 19068 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.345960 19083 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.311740 19046 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.347695 19046 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.319936 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.352156 16596 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:35.352569 19007 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:35.353150 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.353722 16596 raft_consensus.cc:2376] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:35.302424 18915 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:35.324190 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 140739us
W20260709 13:18:35.324720 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 141180us
I20260709 13:18:35.290774 19134 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
W20260709 13:18:35.325115 19130 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:18:35.294205 19125 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:18:35.326458 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.326910 18998 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.312868 19116 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:18:35.329705 18934 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:35.300865 17411 rpcz_store.cc:269] 0709 13:18:33.964560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.995303 (+ 30743us) service_pool.cc:224] Handling call
0709 13:18:35.257125 (+1261822us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.331207 19085 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.331673 18742 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.335753 19064 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.289254 18823 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.337013 18597 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.314594 19074 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:35.307822 18961 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.312001 16634 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:35.350276 19021 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.387074 16741 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:35.388428 18826 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.368446 16708 leader_election.cc:400] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:35.321341 19088 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:35.378326 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.396248 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1776) took 1895 ms. Trace:
I20260709 13:18:35.396477 16596 rpcz_store.cc:276] 0709 13:18:33.500433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.354338 (+1853905us) service_pool.cc:224] Handling call
0709 13:18:35.396200 (+ 41862us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:35.381296 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.823s (SENT)
I20260709 13:18:35.383282 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.390856 16708 leader_election.cc:403] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:35.362206 19129 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:35.397393 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.823s (SENT)
I20260709 13:18:35.397919 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:35.401059 19123 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.401129 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:35.401803 19007 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.402133 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:35.402163 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:35.403002 16739 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:35.394884 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:35.404012 19125 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:35.408151 18547 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.408545 19074 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:35.409559 16588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 160) took 2086 ms (client timeout 2270 ms). Trace:
W20260709 13:18:35.410121 16588 rpcz_store.cc:269] 0709 13:18:33.322800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.322956 (+   156us) service_pool.cc:224] Handling call
0709 13:18:35.409309 (+2086353us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540033a9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:35.409482 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036573952}
I20260709 13:18:35.411171 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.404044 19133 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.411549 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.412621 16588 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:35.413638 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.415094 18826 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.390271 19084 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.417234 18434 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:35.417763 18434 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:35.405527 19126 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:35.368148 18915 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.391161 18961 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.409723 19074 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.508469 19021 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:35.508934 19046 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.414520 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2308) took 1785 ms. Trace:
I20260709 13:18:35.376047 18932 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.021s
W20260709 13:18:35.419845 17374 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.203s (SENT)
I20260709 13:18:35.518045 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:35.519040 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 99253.6us
W20260709 13:18:35.428954 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 117015us
I20260709 13:18:35.520926 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:35.521158 18960 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.522619 18915 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:35.419701 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.523464 18960 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.523967 16587 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.524183 18960 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:35.428332 18681 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:35.522698 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:35.534847 16587 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 3.
I20260709 13:18:35.535493 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1803) took 1056 ms. Trace:
I20260709 13:18:35.411916 16596 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:35.536569 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1795) took 1595 ms. Trace:
W20260709 13:18:35.536931 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.410605 19043 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.535851 19116 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:35.516489 16588 rpcz_store.cc:276] 0709 13:18:33.629388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.413342 (+1783954us) service_pool.cc:224] Handling call
0709 13:18:35.414459 (+  1117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.452924 19126 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.536916 16596 rpcz_store.cc:276] 0709 13:18:33.941639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.396902 (+1455263us) service_pool.cc:224] Handling call
0709 13:18:35.536506 (+139604us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.556865 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.557799 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1801) took 1078 ms. Trace:
I20260709 13:18:35.558188 16596 rpcz_store.cc:276] 0709 13:18:34.478939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.556577 (+1077638us) service_pool.cc:224] Handling call
0709 13:18:35.557730 (+  1153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.430358 18953 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:35.516427 18920 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.537276 19125 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:35.541055 16587 rpcz_store.cc:276] 0709 13:18:34.479478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.419370 (+939892us) service_pool.cc:224] Handling call
0709 13:18:35.535439 (+116069us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.545296 18934 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.545639 19043 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.547163 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.547590 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.565390 16593 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.549130 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.551067 19116 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:35.551208 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.572000 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1783) took 2007 ms. Trace:
I20260709 13:18:35.572422 16592 rpcz_store.cc:276] 0709 13:18:33.565129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.550911 (+1985782us) service_pool.cc:224] Handling call
0709 13:18:35.571941 (+ 21030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.520959 19010 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.491815 18996 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:35.418849 19084 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.551962 19007 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:35.543885 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.590006 16588 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:35.565389 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1790) took 1942 ms. Trace:
I20260709 13:18:35.567556 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.591535 16669 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:35.568193 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.592645 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.590853 16594 rpcz_store.cc:276] 0709 13:18:33.622572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.546846 (+1924274us) service_pool.cc:224] Handling call
0709 13:18:35.565336 (+ 18490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.594002 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.563725 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.569380 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.556895 18932 consensus_queue.cc:1243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:35.570824 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.566817 16589 raft_consensus.cc:2376] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6. Re-sending same reply.
I20260709 13:18:35.602248 16594 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:35.602591 16668 pending_rounds.cc:85] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 3
I20260709 13:18:35.574244 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.552170 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.550658 19089 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.576236 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.418836 18434 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:35.572727 19116 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:35.582046 16662 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 13:18:35.611238 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 207610us
I20260709 13:18:35.613830 19010 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:35.591091 19085 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.583335 18918 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:35.606683 18996 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:35.615262 18918 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:35.616088 18918 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.615984 18996 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.617089 18742 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.617298 18918 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:35.618116 19089 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.589397 19129 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.619230 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.619393 18920 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.619840 16591 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:35.620350 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1807) took 1139 ms. Trace:
I20260709 13:18:35.620674 16591 rpcz_store.cc:276] 0709 13:18:34.480600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.607916 (+1127316us) service_pool.cc:224] Handling call
0709 13:18:35.620290 (+ 12374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.621495 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.621981 16591 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.615274 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:35.624086 16736 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
W20260709 13:18:35.625134 16590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1769) took 2127 ms (client timeout 2451 ms). Trace:
I20260709 13:18:35.584479 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:35.562497 19126 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.586092 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:35.628392 16740 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:35.629426 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:35.629913 16740 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:35.586655 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:35.576246 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.589810 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.595091 16663 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:35.632663 16708 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.549s (SENT)
I20260709 13:18:35.633143 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:35.595423 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.634552 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.636991 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1797) took 1633 ms. Trace:
I20260709 13:18:35.637511 16587 rpcz_store.cc:276] 0709 13:18:34.004156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.634314 (+1630158us) service_pool.cc:224] Handling call
0709 13:18:35.636920 (+  2606us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.638767 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.600028 16593 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
W20260709 13:18:35.602301 16589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1794) took 1676 ms (client timeout 2052 ms). Trace:
I20260709 13:18:35.620483 18961 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:35.606940 16596 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:35.603147 16665 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:35.600824 19083 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.616578 19085 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:35.583636 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:35.686203 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.626495 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.602975 16668 pending_rounds.cc:107] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 13:18:35.687208 16665 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.687553 16669 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:35.688556 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.689594 16664 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:35.569514 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.028s	sys 0.026s
W20260709 13:18:35.639860 16589 rpcz_store.cc:269] 0709 13:18:33.925850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.548889 (+1623039us) service_pool.cc:224] Handling call
0709 13:18:35.602227 (+ 53338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.573424 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.639887 16587 raft_consensus.cc:2368] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
I20260709 13:18:35.641146 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1802) took 1161 ms. Trace:
I20260709 13:18:35.641486 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.615978 18434 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.609890 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.644528 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:35.704043 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.217s	user 0.028s	sys 0.026s
I20260709 13:18:35.704138 16593 rpcz_store.cc:276] 0709 13:18:34.479224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.547181 (+1067957us) service_pool.cc:224] Handling call
0709 13:18:35.641021 (+ 93840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.705365 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.705847 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:35.625526 16590 rpcz_store.cc:269] 0709 13:18:33.498096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.551860 (+2053764us) service_pool.cc:224] Handling call
0709 13:18:35.625080 (+ 73220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.688756 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:35.704573 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 1.21724e+06us
I20260709 13:18:35.684121 16740 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:35.696187 16668 raft_consensus.cc:2917] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:18:35.708532 16668 raft_consensus.cc:2889] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.708966 16663 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:35.709265 16740 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:35.711000 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.710139 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.711673 16593 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:35.711781 16589 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:35.714027 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:35.714335 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 69886.1us
I20260709 13:18:35.714751 16589 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.712328 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1798) took 1697 ms. Trace:
I20260709 13:18:35.716208 16587 rpcz_store.cc:276] 0709 13:18:34.015297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.638445 (+1623148us) service_pool.cc:224] Handling call
0709 13:18:35.712271 (+ 73826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.717273 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.717685 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.719205 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.720163 16663 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:35.717911 16587 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:35.720093 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:35.732164 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:35.640610 19007 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:35.740527 16588 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:35.741844 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.742947 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.743467 16590 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:35.744205 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.734719 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.745338 16590 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:35.746060 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1773) took 2246 ms. Trace:
I20260709 13:18:35.746425 16590 rpcz_store.cc:276] 0709 13:18:33.499645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.744937 (+2245292us) service_pool.cc:224] Handling call
0709 13:18:35.745177 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.745689 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.746003 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.748077 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2311) took 2063 ms. Trace:
I20260709 13:18:35.737649 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.749123 16663 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:35.747953 16738 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:35.750824 16590 rpcz_store.cc:276] 0709 13:18:33.711735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.747282 (+2035547us) service_pool.cc:224] Handling call
0709 13:18:35.747555 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.747759 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.747785 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:35.747834 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:35.747866 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.747924 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:35.748027 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.751757 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:35.752635 17415 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:35.753479 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.754493 16590 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:35.736256 16591 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:35.755154 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:35.749912 16665 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:35.756549 16591 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:18:35.757032 16591 pending_rounds.cc:85] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 2
I20260709 13:18:35.755367 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.757318 16591 pending_rounds.cc:107] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:18:35.758153 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.758303 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2314) took 2027 ms. Trace:
I20260709 13:18:35.759233 16590 rpcz_store.cc:276] 0709 13:18:33.731192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.752347 (+2021155us) service_pool.cc:224] Handling call
0709 13:18:35.752704 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.758052 (+  5348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.758235 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.760517 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.761289 16591 raft_consensus.cc:2889] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:35.762001 19123 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:35.764161 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.758889 16596 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:35.764415 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:35.767383 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1806) took 1287 ms. Trace:
I20260709 13:18:35.767444 16665 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:18:35.767808 16596 rpcz_store.cc:276] 0709 13:18:34.480321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.711889 (+1231568us) service_pool.cc:224] Handling call
0709 13:18:35.767331 (+ 55442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.768920 16596 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:35.769260 16596 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:35.769322 16587 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:35.769338 16665 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:35.775024 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.807433 16663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:35.734797 19083 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:35.809935 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1751) took 1264 ms. Trace:
I20260709 13:18:35.810273 16664 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:35.810401 16742 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 923) took 2438 ms (client timeout 2064 ms). Trace:
I20260709 13:18:35.769979 16596 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:35.811318 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2324) took 2076 ms. Trace:
I20260709 13:18:35.894433 16669 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:35.813799 16663 raft_consensus.cc:606] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:35.816087 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.894770 16591 rpcz_store.cc:276] 0709 13:18:33.734454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.755209 (+2020755us) service_pool.cc:224] Handling call
0709 13:18:35.756149 (+   940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.808824 (+ 52675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.811243 (+  2419us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:35.734637 18934 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:35.734987 18915 raft_consensus.cc:2764] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:18:35.816285 16671 rpcz_store.cc:276] 0709 13:18:34.545712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.545833 (+   121us) service_pool.cc:224] Handling call
0709 13:18:35.806270 (+1260437us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b540033ac54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:35.809868 (+  3598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154219776}
I20260709 13:18:35.812965 16667 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:18:35.829186 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:35.758638 18920 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:35.692512 18972 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
W20260709 13:18:35.921064 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:35.816416 16742 rpcz_store.cc:269] 0709 13:18:33.371825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.380256 (+  8431us) service_pool.cc:224] Handling call
0709 13:18:35.807013 (+2426757us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400162e54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:35.810331 (+  3318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3025038592}
I20260709 13:18:35.909422 16665 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:35.894482 16596 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:35.915079 16589 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:35.741158 18804 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260709 13:18:35.924986 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.742092 19148 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.169s
I20260709 13:18:35.926723 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.927961 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.929423 16664 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:35.941259 16665 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:35.933082 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.754766 19085 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.159s
I20260709 13:18:35.944188 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:35.944983 16589 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:35.763144 18742 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.155s
I20260709 13:18:35.946583 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:35.916445 19064 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.024704 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:35.764703 19084 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.162s
I20260709 13:18:36.024204 16589 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:36.025803 16742 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:36.030203 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.031019 16589 raft_consensus.cc:2376] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7. Re-sending same reply.
I20260709 13:18:36.032066 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.033632 16589 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:36.028092 16667 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:36.021005 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:36.028304 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.131848 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:36.132086 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.132683 16589 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:36.133594 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.134620 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.136201 16589 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:36.137702 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.139045 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.140012 16589 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:36.175379 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.176208 16589 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.177035 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:36.178208 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:36.179682 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:35.945982 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:36.185693 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:36.192989 16589 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:36.197036 16589 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 13:18:36.197760 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2330) took 1421 ms. Trace:
I20260709 13:18:36.198130 16589 rpcz_store.cc:276] 0709 13:18:34.776089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.196343 (+1420254us) service_pool.cc:224] Handling call
0709 13:18:36.196723 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:36.197484 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.197718 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.200572 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2331) took 1424 ms. Trace:
W20260709 13:18:36.185817 19156 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:18:36.201126 16589 rpcz_store.cc:276] 0709 13:18:34.776329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.199481 (+1423152us) service_pool.cc:224] Handling call
0709 13:18:36.199967 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.200180 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.200215 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.200266 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:36.200308 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.200373 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.200504 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.203595 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:36.205050 16664 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:36.237829 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2325) took 2503 ms. Trace:
W20260709 13:18:36.240270 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.259699 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2320) took 2526 ms. Trace:
I20260709 13:18:36.260169 16593 rpcz_store.cc:276] 0709 13:18:33.733242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.754358 (+2021116us) service_pool.cc:224] Handling call
0709 13:18:35.811825 (+ 57467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.259372 (+447547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.259400 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.259438 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.259467 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.259528 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.259633 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100898,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":22786,"thread_start_us":446965,"threads_started":1}
W20260709 13:18:36.261731 16736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.016s	sys 0.030s
W20260709 13:18:36.262158 16736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.016s	sys 0.030s
I20260709 13:18:36.238560 16588 rpcz_store.cc:276] 0709 13:18:33.734764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.758585 (+2023821us) service_pool.cc:224] Handling call
0709 13:18:35.758845 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.759069 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:18:35.762757 (+  3688us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.237369 (+474612us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.237446 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:36.237511 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.237600 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.237752 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10571,"prepare.queue_time_us":11907,"prepare.run_cpu_time_us":4442,"prepare.run_wall_time_us":21091,"raft.queue_time_us":31121,"thread_start_us":463521,"threads_started":1}
I20260709 13:18:36.293678 16588 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:36.294271 16588 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:36.315200 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:36.340101 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51343.2us
I20260709 13:18:36.341552 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:36.207726 17413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 276) took 1644 ms (client timeout 2095 ms). Trace:
I20260709 13:18:36.205312 17409 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:36.240118 19016 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:36.383904 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:36.390843 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:36.396057 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.342537 16588 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:36.401418 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:36.410583 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:36.317965 19152 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:36.372123 18961 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:36.402938 19154 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.707s	user 0.000s	sys 0.001s
I20260709 13:18:36.304026 18916 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:18:36.317615 18804 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 13:18:36.404014 18934 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:36.459262 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:36.467530 16671 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:36.463485 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 463) took 3368 ms. Trace:
W20260709 13:18:36.479988 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.483690 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:36.515077 16593 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:18:36.371764 17413 rpcz_store.cc:269] 0709 13:18:34.560759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.565378 (+  4619us) service_pool.cc:224] Handling call
0709 13:18:36.194278 (+1628900us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400592254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:36.204876 (+ 10598us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521166208}
I20260709 13:18:36.458546 19152 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.457218 19085 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.491824 18934 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:36.385061 19129 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:18:36.492352 18915 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:36.495034 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.497387 18898 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.498813 18916 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:36.386806 19144 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
W20260709 13:18:36.512953 16596 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.589s	user 0.006s	sys 0.002s
I20260709 13:18:36.521677 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.503002 19084 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.525842 16596 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:18:36.526298 16596 pending_rounds.cc:85] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 6
I20260709 13:18:36.526628 16596 pending_rounds.cc:107] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:18:36.527020 16596 raft_consensus.cc:2889] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:36.527760 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1804) took 2048 ms. Trace:
I20260709 13:18:36.528008 16596 rpcz_store.cc:276] 0709 13:18:34.479739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.768549 (+1288810us) service_pool.cc:224] Handling call
0709 13:18:35.768817 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.527588 (+758771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.527721 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.529855 16743 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.531214 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:36.531060 16590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.003s	sys 0.024s
I20260709 13:18:36.531927 16743 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:36.532022 16590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.003s	sys 0.024s
W20260709 13:18:36.533697 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.007s	sys 0.015s
W20260709 13:18:36.534205 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.007s	sys 0.015s
W20260709 13:18:36.535478 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 771064us
W20260709 13:18:36.520890 19161 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:18:36.518838 19164 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:18:36.518253 18742 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
W20260709 13:18:36.521373 18972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.013s	sys 0.021s
W20260709 13:18:36.520668 19160 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:18:36.523253 19155 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.390s	user 0.000s	sys 0.001s
I20260709 13:18:36.603067 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 472) took 1767 ms. Trace:
I20260709 13:18:36.603533 17410 rpcz_store.cc:276] 0709 13:18:34.835322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:34.972660 (+137338us) service_pool.cc:224] Handling call
0709 13:18:35.572987 (+600327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.620053 (+ 47066us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.620085 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.620532 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:18:35.917944 (+297412us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:35.931650 (+ 13706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:35.943356 (+ 11706us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.600895 (+657539us) raft_consensus.cc:1685] finished
0709 13:18:36.600915 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.602760 (+  1845us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400483850. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:36.603000 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139,"raft.queue_time_us":13189,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":16987,"spinlock_wait_cycles":1374891776,"thread_start_us":274744,"threads_started":1,"wal-append.queue_time_us":574147}
I20260709 13:18:36.571843 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2326) took 2701 ms. Trace:
W20260709 13:18:36.552392 19074 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.045s	user 0.007s	sys 0.003s
I20260709 13:18:36.584174 16593 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:36.558835 19099 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:36.550998 18916 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:36.523741 19085 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.548929 19084 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.592868 16743 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:36.519726 18915 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:36.519421 18934 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:36.599650 16595 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1799) took 2317 ms (client timeout 1801 ms). Trace:
W20260709 13:18:36.624259 16595 rpcz_store.cc:269] 0709 13:18:34.282619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.599416 (+2316797us) service_pool.cc:211] Skipping call since client already timed out
0709 13:18:36.599567 (+   151us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:18:36.554899 18918 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.937s	user 0.004s	sys 0.006s
I20260709 13:18:36.540441 16671 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:36.606895 16590 rpcz_store.cc:276] 0709 13:18:33.870851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.760494 (+1889643us) service_pool.cc:224] Handling call
0709 13:18:35.760842 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.761090 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:36.021403 (+260313us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.571247 (+549844us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.571336 (+    89us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.571405 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.571512 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.571752 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45508,"prepare.queue_time_us":175019,"prepare.run_cpu_time_us":13935,"prepare.run_wall_time_us":77219,"raft.queue_time_us":38214,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5551,"thread_start_us":704637,"threads_started":2}
I20260709 13:18:36.635248 16590 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:36.635813 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2341) took 1437 ms. Trace:
I20260709 13:18:36.636112 16590 rpcz_store.cc:276] 0709 13:18:35.198384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.634673 (+1436289us) service_pool.cc:224] Handling call
0709 13:18:36.635088 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.635610 (+   522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.635765 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:36.556977 18434 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.003s
I20260709 13:18:36.529103 18898 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.645005 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.615401 16593 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:36.646015 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2335) took 1868 ms. Trace:
I20260709 13:18:36.646090 16590 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:18:36.553766 18961 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:18:36.591912 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.458s (SENT)
I20260709 13:18:36.646502 16559 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:36.646659 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2342) took 1445 ms. Trace:
I20260709 13:18:36.647318 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:36.647703 16590 rpcz_store.cc:276] 0709 13:18:35.201401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.636745 (+1435344us) service_pool.cc:224] Handling call
0709 13:18:36.636965 (+   220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.646444 (+  9479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.646612 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.612442 19144 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.631593 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2337) took 1853 ms. Trace:
I20260709 13:18:36.632575 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2282) took 1047 ms. Trace:
I20260709 13:18:36.610942 18742 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:18:36.637971 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:36.652434 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60547.8us
I20260709 13:18:36.642568 16594 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:36.514420 17412 rpcz_store.cc:276] 0709 13:18:33.095353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:33.096360 (+  1007us) service_pool.cc:224] Handling call
0709 13:18:33.130335 (+ 33975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.021864 (+1891529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.463136 (+1441272us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400091854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:36.463380 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3842660736}
I20260709 13:18:36.646384 16593 rpcz_store.cc:276] 0709 13:18:34.777327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.609915 (+1832588us) service_pool.cc:224] Handling call
0709 13:18:36.612124 (+  2209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.645782 (+ 33658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.645964 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.621369 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.649827 16595 rpcz_store.cc:276] 0709 13:18:34.777855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.630624 (+1852769us) service_pool.cc:224] Handling call
0709 13:18:36.630987 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.631203 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.631234 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.631273 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.631305 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.631377 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.631523 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.649940 16745 rpcz_store.cc:276] 0709 13:18:35.585477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.631492 (+ 46015us) service_pool.cc:224] Handling call
0709 13:18:35.691403 (+ 59911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.691697 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:35.919933 (+228236us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.626238 (+706305us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b1800458bc0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f32835dc3c7 00007f327fbee9a7 00007f327fbfe0bf 00007f327fbfe038
0709 13:18:36.631792 (+  5554us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.632075 (+   283us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.632142 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.632198 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.632372 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.632506 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10629,"prepare.queue_time_us":956049,"raft.queue_time_us":5932,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":11480,"spinlock_wait_cycles":6901120,"thread_start_us":211861,"threads_started":1,"wal-append.queue_time_us":22331}
I20260709 13:18:36.659278 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2279) took 1074 ms. Trace:
I20260709 13:18:36.623976 19074 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.638402 18915 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:36.661690 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.681209 18916 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.682018 18998 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.659631 16737 rpcz_store.cc:276] 0709 13:18:35.584701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.587021 (+  2320us) service_pool.cc:224] Handling call
0709 13:18:35.587262 (+   241us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:35.601214 (+ 13952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.601242 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:35.611119 (+  9877us) log.cc:844] Serialized 52 byte log entry
0709 13:18:35.915608 (+304489us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:36.454037 (+538429us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.454363 (+   326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.454466 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.659083 (+204617us) raft_consensus.cc:1685] finished
0709 13:18:36.659101 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.659225 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":741559,"raft.queue_time_us":339,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":15340,"thread_start_us":715798,"threads_started":2,"wal-append.queue_time_us":698923}
I20260709 13:18:36.652426 18742 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:36.648643 18804 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.666460 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:36.667127 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:36.667475 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:36.668399 19152 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:36.670152 16593 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:36.703538 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2350) took 1173 ms. Trace:
I20260709 13:18:36.703871 16593 rpcz_store.cc:276] 0709 13:18:35.530582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.657549 (+1126967us) service_pool.cc:224] Handling call
0709 13:18:36.669873 (+ 12324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.703299 (+ 33426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.703480 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.652061 18918 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:36.667210 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:36.672107 19176 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:36.667215 18898 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:36.648716 18961 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.656955 18920 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:36.638154 18934 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:36.674775 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.676296 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.708963 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.683501 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.660745 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2351) took 1129 ms. Trace:
I20260709 13:18:36.684729 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:36.691771 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:36.695855 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 285347us
I20260709 13:18:36.716917 16595 rpcz_store.cc:276] 0709 13:18:35.530819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.658758 (+1127939us) service_pool.cc:224] Handling call
0709 13:18:36.660293 (+  1535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.660470 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.660494 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.660533 (+    39us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:36.660562 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.660613 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.660697 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:36.717789 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.696973 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1768) took 1062 ms. Trace:
I20260709 13:18:36.649108 19165 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
W20260709 13:18:36.696229 19126 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.011s	user 0.007s	sys 0.003s
I20260709 13:18:36.656369 19144 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.657760 18951 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:18:36.657927 19129 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:18:36.727715 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:36.728317 16737 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.728590 16743 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.732628 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.733453 16743 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:36.683033 19053 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:18:36.737869 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.739537 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.741514 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:36.706833 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:36.745174 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.753144 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.753358 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.754091 16743 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:36.710599 18918 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.757264 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.684319 19074 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.734797 18898 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:36.735119 19044 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.760783 16737 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.745417 19099 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:36.759958 18918 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.760988 17410 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:36.772526 18951 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:18:36.774168 19168 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:18:36.775130 19177 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:18:36.749001 19085 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:36.778187 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:36.795967 16594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1796) took 2810 ms (client timeout 3098 ms). Trace:
W20260709 13:18:36.796525 16594 rpcz_store.cc:269] 0709 13:18:33.985942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.609666 (+1623724us) service_pool.cc:224] Handling call
0709 13:18:36.788530 (+1178864us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b540010d954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:36.795905 (+  7375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142624128}
I20260709 13:18:36.755115 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:36.798373 16743 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:36.798754 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 140009us
I20260709 13:18:36.781826 19099 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:36.751770 18998 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:36.782507 16590 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:36.789069 16589 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:36.737010 19126 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.789783 19176 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:36.787034 18951 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:36.790516 19181 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:36.811574 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 150850us
I20260709 13:18:36.801069 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:36.779340 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 72982.5us
W20260709 13:18:36.821594 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:36.819746 18998 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:36.759524 18742 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.825397 18742 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.825968 19144 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:36.826505 19144 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:36.826830 19144 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:36.779441 19044 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.822958 18998 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.828127 16745 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.800185 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:36.830058 16743 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:36.846391 16671 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:36.820822 19181 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:36.848433 19181 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.815096 19099 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:36.852504 19044 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:36.852854 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2334) took 2075 ms. Trace:
I20260709 13:18:36.853351 16596 rpcz_store.cc:276] 0709 13:18:34.777086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.528616 (+1751530us) service_pool.cc:224] Handling call
0709 13:18:36.528840 (+   224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:36.571312 (+ 42472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:36.571363 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:36.572014 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.780668 (+208654us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.780717 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.780801 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.852623 (+ 71822us) raft_consensus.cc:1685] finished
0709 13:18:36.852649 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.852786 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22193,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":174394,"threads_started":1,"wal-append.queue_time_us":226529}
I20260709 13:18:36.815541 19084 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:36.855252 18961 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:36.827636 19144 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.856717 19144 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:36.860761 19126 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:36.862080 19074 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:36.865940 19116 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.241s	user 0.006s	sys 0.002s
I20260709 13:18:36.865945 19099 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:36.813573 18951 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.867990 18951 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:36.868451 19099 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:36.873838 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1094) took 1299 ms. Trace:
W20260709 13:18:36.874511 19151 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.153s	user 0.009s	sys 0.011s
I20260709 13:18:36.874293 16668 rpcz_store.cc:276] 0709 13:18:35.574181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.574298 (+   117us) service_pool.cc:224] Handling call
0709 13:18:35.602410 (+ 28112us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.743157 (+140747us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.743218 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.743741 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.202365 (+458624us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.202402 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.202468 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:36.873592 (+671124us) raft_consensus.cc:1685] finished
0709 13:18:36.873614 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.873764 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":284454,"threads_started":1,"wal-append.queue_time_us":721409}
I20260709 13:18:36.846417 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:36.883898 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:36.887529 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 140111us
W20260709 13:18:36.892839 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46527.2us
I20260709 13:18:36.893824 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.899608 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:36.906643 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:36.907347 16668 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:36.880383 18916 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:36.916855 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b778f817264440987b35333e6540c72"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:36.917745 16671 raft_consensus.cc:606] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.918184 16671 raft_consensus.cc:614] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:36.922477 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:36.923367 16667 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:36.932183 16670 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 13:18:36.866564 19116 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.904078 18961 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:36.909355 18742 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:36.906559 19084 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:36.937036 16744 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:18:36.938161 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 220501us
I20260709 13:18:36.912712 18916 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:36.950562 16744 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:36.952193 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.954843 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:36.959255 16738 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:36.938742 19085 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:36.968276 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:36.967077 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 1.33245e+06us
I20260709 13:18:36.988231 16561 leader_election.cc:400] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5.
I20260709 13:18:36.988575 16561 leader_election.cc:403] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:36.989439 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:36.954156 19165 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.962452 19189 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:36.969769 19181 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:36.962688 18742 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:37.021201 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.017670 16671 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.025609 16663 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:37.018055 19196 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.023046 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.032022 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.032914 16666 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:37.020412 16668 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.021720 16662 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:18:37.020954 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.022531 19084 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:37.034164 16665 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:36.987762 19126 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.018116 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 95538.2us
W20260709 13:18:37.020526 16708 leader_election.cc:336] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.801s (SENT)
I20260709 13:18:37.036144 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.060743 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:37.061498 16708 leader_election.cc:336] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.918s (SENT)
I20260709 13:18:37.061771 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:18:37.062595 16708 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.044s (SENT)
I20260709 13:18:37.060319 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:37.063953 16708 leader_election.cc:336] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.049s (SENT)
W20260709 13:18:37.064505 17374 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.677s (SENT)
W20260709 13:18:37.065768 16708 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.052s (SENT)
I20260709 13:18:36.963147 19198 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
I20260709 13:18:37.066342 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:18:37.067826 16708 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.151s (SENT)
W20260709 13:18:37.069252 16708 leader_election.cc:336] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.235s (SENT)
I20260709 13:18:37.069823 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:36.954380 19070 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.017944 19201 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:36.990942 19085 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.031431 19196 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.071952 18953 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.064085 19016 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:37.075421 16708 leader_election.cc:336] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.677s (SENT)
I20260709 13:18:37.086186 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:37.066212 19044 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:18:37.079496 19194 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:18:37.080264 19169 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.560s	user 0.000s	sys 0.004s
I20260709 13:18:37.072166 19067 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.016630 18916 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.035202 19126 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.032136 18915 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.024634 19189 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.035053 19084 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:37.102721 19205 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:18:37.103740 19197 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 13:18:37.118711 19133 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.119220 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:37.122957 19199 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:18:37.106827 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.095484 19201 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.098220 19016 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.087870 19070 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.087363 19196 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.134212 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 73976us
I20260709 13:18:37.099030 19165 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:37.123628 19210 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.143079 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:37.120846 19211 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.144119 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.144606 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.124912 18916 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:37.148000 16634 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.604s (SENT)
I20260709 13:18:37.155051 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:37.137221 19067 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:18:37.157675 16737 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:37.133533 19129 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:37.160501 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.100476 18742 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.156025 19218 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.179586 18934 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.179963 19044 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:18:37.182194 19163 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.682s	user 0.001s	sys 0.002s
I20260709 13:18:37.158646 19116 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.158156 19211 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:37.195801 19211 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.189834 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:37.194847 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2281) took 1609 ms. Trace:
I20260709 13:18:37.177922 18953 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.198925 19085 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.164912 18998 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:18:37.201016 16737 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:37.200563 16736 rpcz_store.cc:276] 0709 13:18:35.585255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.624622 (+ 39367us) service_pool.cc:224] Handling call
0709 13:18:35.657591 (+ 32969us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:35.706916 (+ 49325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:35.706951 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:35.752272 (+ 45321us) log.cc:844] Serialized 26 byte log entry
0709 13:18:36.262578 (+510306us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:36.262636 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.262747 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:37.181913 (+919166us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b240022ae88. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327c97d9c4 00007f327c97bacb 00007f327cb36dc9 00007f327cb36a55 00007f327cb3609d 00007f327f76810e 00007f327f77f88c 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221
0709 13:18:37.194502 (+ 12589us) raft_consensus.cc:1685] finished
0709 13:18:37.194522 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.194769 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23335296,"thread_start_us":509895,"threads_started":1,"wal-append.queue_time_us":681317}
I20260709 13:18:37.206192 17413 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:37.207819 17411 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:37.191576 19044 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:37.215168 19125 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:37.205539 17409 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:37.164820 19210 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:37.192276 19218 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:37.237879 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 162516us
I20260709 13:18:37.201651 19208 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:37.221999 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.242224 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 206168us
W20260709 13:18:37.250113 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0904368259998591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 13:18:37.256455 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:37.256567 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0512560670001676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3098s for a response
W20260709 13:18:37.257876 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0413560670001516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3109s for a response
W20260709 13:18:37.258466 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0377560670001458s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3113s for a response
I20260709 13:18:37.219918 19125 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:37.259856 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:37.260818 18951 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:37.261410 18951 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:37.261950 18951 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:37.261023 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:37.211766 19181 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.261111 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.210333 18953 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.257583 19126 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:37.263381 19053 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:37.247167 18918 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.244853 19044 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.291463 19044 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.283418 19189 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.241124 19224 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:18:37.279290 19083 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:37.281201 19221 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.300356 19231 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.301497 19218 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.302114 19218 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.302593 19218 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.285910 18996 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:37.279520 19126 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:18:37.314095 19046 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:37.315510 19167 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.824s	user 0.002s	sys 0.000s
W20260709 13:18:37.317790 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.320459 19099 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:37.321779 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.321880 19099 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:37.322471 19099 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:37.323866 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.324824 19099 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:37.310258 19153 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 1.391s	user 0.013s	sys 0.017s
I20260709 13:18:37.311436 17420 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:37.327797 18953 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:37.326614 19099 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.298676 19083 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:37.323200 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:37.329627 16744 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:37.329775 19083 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.330792 19083 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.323885 18918 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.331651 19176 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:37.332049 19176 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:37.332232 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:37.332931 17409 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:37.332867 19176 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.334121 19176 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.334604 19211 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.335399 19231 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.336453 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:37.336784 19211 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.337260 18953 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:37.338718 18915 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.323449 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:37.271421 19125 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.343010 19173 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.345139 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:37.355786 19210 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.362617 19201 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5.
I20260709 13:18:37.343897 18996 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.367475 17412 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.338773 18953 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.378074 18953 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.378655 18953 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.379712 19218 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.360771 19046 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:37.383167 19187 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.569s	user 0.000s	sys 0.003s
W20260709 13:18:37.383471 19158 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 1.488s	user 0.010s	sys 0.013s
W20260709 13:18:37.383975 19183 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.568s	user 0.000s	sys 0.002s
I20260709 13:18:37.375150 19125 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.385944 18996 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:37.360468 19173 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.376278 18951 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.344962 19070 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:37.394627 19070 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.395900 19070 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.400133 19178 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:37.400709 19178 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:37.400554 19196 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.401572 19178 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.402359 19173 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.402530 19178 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:37.402691 19196 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.277832 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:37.437831 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 159899us
W20260709 13:18:37.440321 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.441119 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.442629 16592 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:37.445879 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.447548 16596 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:37.448804 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.449924 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.450349 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 189335us
I20260709 13:18:37.452158 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:37.459556 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 112828us
I20260709 13:18:37.460021 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.461795 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:37.360911 18898 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:18:37.483884 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:37.485275 19212 raft_consensus.cc:1064] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:37.487587 18934 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.452293 16667 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.466845 19046 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:37.467114 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.470261 18951 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.471692 19176 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.472652 19180 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:37.494191 19180 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.474152 19232 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.496498 19046 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.498080 19232 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.499265 19233 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.501940 17410 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:37.504930 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2329) took 2729 ms. Trace:
I20260709 13:18:37.508764 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2328) took 2733 ms. Trace:
I20260709 13:18:37.509717 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:36.718801 16669 rpcz_store.cc:276] 0709 13:18:35.635023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.914151 (+279128us) service_pool.cc:224] Handling call
0709 13:18:35.914456 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.914689 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:36.518701 (+604012us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.696583 (+177882us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.696655 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:36.696729 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:36.696792 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:36.696917 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17561,"prepare.queue_time_us":290207,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":4078,"raft.queue_time_us":84,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5039,"thread_start_us":676564,"threads_started":2}
I20260709 13:18:37.477384 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.501446 19232 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.467149 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2332) took 2690 ms. Trace:
I20260709 13:18:37.515476 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.515727 16589 rpcz_store.cc:276] 0709 13:18:34.776592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.205841 (+1429249us) service_pool.cc:224] Handling call
0709 13:18:36.788696 (+582855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.812285 (+ 23589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.446521 (+634236us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b540010d950. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:37.466832 (+ 20311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339664768}
I20260709 13:18:37.516774 16745 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:37.522995 18951 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:37.523653 16738 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.523972 18951 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.526710 18951 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.527428 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:37.528303 16742 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:37.533201 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:37.534057 16737 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.536146 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.538286 19233 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:37.539209 19233 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:37.544615 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:37.546802 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.548273 16665 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:37.551726 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.564704 16744 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:37.505457 16591 rpcz_store.cc:276] 0709 13:18:34.775820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.909034 (+1133214us) service_pool.cc:224] Handling call
0709 13:18:35.928159 (+ 19125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:35.928924 (+   765us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:35.936429 (+  7505us) log.cc:844] Serialized 14 byte log entry
0709 13:18:37.030063 (+1093634us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:37.503540 (+473477us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b1800396820. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327f78c4bc 00007f327cada48b 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f78423a
0709 13:18:37.504086 (+   546us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:37.504187 (+   101us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:37.504375 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.504538 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.504731 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34346,"prepare.queue_time_us":266044,"prepare.run_cpu_time_us":3086,"prepare.run_wall_time_us":11133,"spinlock_wait_cycles":56739840,"thread_start_us":322570,"threads_started":1,"wal-append.queue_time_us":523334}
I20260709 13:18:37.569211 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2348) took 2039 ms. Trace:
I20260709 13:18:37.569702 16590 rpcz_store.cc:276] 0709 13:18:35.530085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.648695 (+1118610us) service_pool.cc:224] Handling call
0709 13:18:36.649011 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.812237 (+163226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.568703 (+756466us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54003b74d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:37.569013 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306021760}
I20260709 13:18:37.571333 16741 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:37.576066 16588 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.585042 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:37.585659 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:37.586731 19180 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.586843 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:37.586089 18908 raft_consensus.cc:1064] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:18:37.595432 16592 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2343) took 2360 ms (client timeout 1716 ms). Trace:
I20260709 13:18:37.596208 16665 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:37.595939 16592 rpcz_store.cc:269] 0709 13:18:35.235230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.573112 (+337882us) service_pool.cc:224] Handling call
0709 13:18:37.595150 (+2022038us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b5400339fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:37.595326 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3987229568}
I20260709 13:18:37.598665 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:37.608115 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.610378 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:37.611459 17412 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.612744 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.613390 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.614600 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.615134 19143 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:37.615427 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 131913us
I20260709 13:18:37.498498 19212 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.618906 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.619465 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:37.689314 16669 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.660981 19143 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.694509 19143 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.619859 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.620365 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.621234 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.696944 17412 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:37.626354 19099 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.698025 19099 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.700958 19099 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:37.701426 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:37.540964 19233 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.629910 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.630451 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.630597 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.630853 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.631537 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.633157 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.634956 18804 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:37.633193 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.636788 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.637109 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:37.594174 18908 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.509322 16587 rpcz_store.cc:276] 0709 13:18:34.775508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:35.769944 (+994436us) service_pool.cc:224] Handling call
0709 13:18:35.770217 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:36.021005 (+250788us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:36.521463 (+500458us) log.cc:844] Serialized 14 byte log entry
0709 13:18:36.699713 (+178250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:36.699778 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:36.699833 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.383389 (+683556us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:37.508211 (+124822us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400190fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:37.508280 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:37.508530 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":283384,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":10044,"raft.queue_time_us":61565,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":14662,"spinlock_wait_cycles":583076608,"thread_start_us":512306,"threads_started":2,"wal-append.queue_time_us":268006}
W20260709 13:18:37.656476 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.665926 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 51538.2us
I20260709 13:18:37.675287 19173 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:37.673501 19070 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.677820 19180 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.685715 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.730397 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.730652 16589 raft_consensus.cc:2410] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:37.729525 16665 raft_consensus.cc:1217] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Deduplicated request from leader. Original: 1.2->[3.3-5.4]   Dedup: 5.4->[]
I20260709 13:18:37.739589 16665 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 13:18:37.740703 16596 raft_consensus.cc:2410] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:37.740526 19180 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.742071 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:37.743544 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 106769us
I20260709 13:18:37.743631 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.745244 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.746212 16589 raft_consensus.cc:2410] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:18:37.747141 16590 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:37.748123 19180 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.748337 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.749899 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.750519 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.751334 16589 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:37.751570 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.752408 18953 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.752995 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.753945 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.754034 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.754717 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.755062 16593 raft_consensus.cc:2368] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:18:37.755654 16592 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.755934 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.756188 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:37.757244 16593 raft_consensus.cc:2410] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:37.757437 16590 raft_consensus.cc:2410] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:37.757977 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.754027 16740 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:37.758809 16593 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:37.746531 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.270s (SENT)
I20260709 13:18:37.758801 16590 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.755016 16587 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:37.763631 16596 raft_consensus.cc:2410] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:37.753757 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:37.765910 16737 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:37.764858 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:37.761674 19173 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:37.770205 16742 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:37.774394 19044 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:37.776224 19178 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:37.776778 19178 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:37.778446 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.783439 16595 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:37.785974 16493 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 139) took 8464 ms. Trace:
I20260709 13:18:37.787365 19173 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:37.787669 18953 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.788223 16740 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:37.788632 19173 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.778133 19178 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.792109 19178 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:37.789233 17413 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:37.795667 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:37.796826 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50370.1us
I20260709 13:18:37.798427 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:37.798962 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.800877 16588 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:37.800561 16737 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:37.795037 16594 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:37.803289 19147 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
W20260709 13:18:37.802135 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 72580.4us
I20260709 13:18:37.804491 16589 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:37.805174 16669 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:37.803776 18953 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:37.798748 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.810794 18953 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:37.811586 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:37.812281 16743 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:37.812143 18953 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.815595 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2333) took 3038 ms. Trace:
I20260709 13:18:37.816334 16588 rpcz_store.cc:276] 0709 13:18:34.776846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:36.293109 (+1516263us) service_pool.cc:224] Handling call
0709 13:18:36.293505 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:37.575209 (+1281704us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b58003c14d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:18:37.801320 (+226111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:37.815519 (+ 14199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391249792}
I20260709 13:18:37.806695 18804 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:37.826686 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.829288 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:37.827812 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.815210 19046 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.828033 18742 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:37.825680 17412 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.842950 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44289.6us
I20260709 13:18:37.839710 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:37.786500 16493 rpcz_store.cc:276] 0709 13:18:29.265366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:29.265744 (+   378us) service_pool.cc:224] Handling call
0709 13:18:37.702948 (+8437204us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:35.386448 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:35.387082 (+   634us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:35.387118 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:18:35.387148 (+    30us) tablet.cc:662] Decoding operations
0709 13:18:35.388253 (+  1105us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:35.388311 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:35.388339 (+    28us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:18:35.389276 (+   937us) tablet.cc:701] Row locks acquired
0709 13:18:35.389294 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:18:35.389433 (+   139us) write_op.cc:270] Start()
0709 13:18:35.389542 (+   109us) write_op.cc:276] Timestamp: P: 1783603115389387 usec, L: 0
0709 13:18:35.389564 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:18:35.390270 (+   706us) log.cc:844] Serialized 5104 byte log entry
0709 13:18:36.778789 (+1388519us) op_driver.cc:464] REPLICATION: finished
0709 13:18:37.031494 (+252705us) write_op.cc:301] APPLY: starting
0709 13:18:37.063142 (+ 31648us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:37.068365 (+  5223us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:37.080619 (+ 12254us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:37.194650 (+114031us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:37.205415 (+ 10765us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:18:37.210310 (+  4895us) write_op.cc:312] APPLY: finished
0709 13:18:37.244977 (+ 34667us) log.cc:844] Serialized 89 byte log entry
0709 13:18:37.245215 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:37.245367 (+   152us) write_op.cc:454] Released schema lock
0709 13:18:37.245415 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:37.245773 (+   358us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:37.245802 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2936954,"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":227284,"apply.run_cpu_time_us":4839,"apply.run_wall_time_us":223501,"num_ops":12,"prepare.queue_time_us":172211,"prepare.run_cpu_time_us":36280,"prepare.run_wall_time_us":363397,"raft.queue_time_us":331181,"raft.run_cpu_time_us":28839,"raft.run_wall_time_us":212990,"replication_time_us":1371973,"thread_start_us":796987,"threads_started":4,"wal-append.queue_time_us":524802}]]}
I20260709 13:18:37.835347 18742 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:37.866567 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.834970 18951 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:37.867944 18742 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.868152 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:37.874212 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48891us
I20260709 13:18:37.869156 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:37.877326 19046 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.868636 19180 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:18:37.889017 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54833.9us
I20260709 13:18:37.889627 18934 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:37.868019 18804 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.876132 16494 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 3, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:37.903247 16592 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
W20260709 13:18:37.908135 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:37.910055 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.910310 18934 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:37.913118 19212 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:37.913861 18934 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.900686 19070 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.887612 19044 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:18:37.886117 18951 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:37.918490 16669 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:37.886603 19143 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:37.913956 18996 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
W20260709 13:18:37.922439 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57078.9us
I20260709 13:18:37.923098 18996 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:37.920897 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:37.923593 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:37.929033 16739 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:37.923301 16494 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:37.921957 18826 raft_consensus.cc:2764] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:37.938376 19218 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:18:37.943414 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 103476us
I20260709 13:18:37.927776 18996 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:37.950290 16739 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:37.945358 18804 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.955631 17411 raft_consensus.cc:1217] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 LEARNER]: Deduplicated request from leader. Original: 3.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 13:18:37.956493 17411 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 LEARNER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:37.958714 17415 pending_rounds.cc:85] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 3
I20260709 13:18:37.959215 17415 pending_rounds.cc:107] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.4
I20260709 13:18:37.959702 17415 raft_consensus.cc:2917] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Aborting config change with OpId 3.4: Aborted: Op aborted by new leader
I20260709 13:18:37.960271 17415 raft_consensus.cc:2889] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:37.961012 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.961046 18951 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:37.962455 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.963217 17414 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:37.963514 16664 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:37.968150 19143 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:37.970018 17410 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:37.973625 19232 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:37.974186 19232 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:37.974576 19232 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:37.976938 19245 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:18:37.981685 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:37.936024 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:37.999197 16665 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:38.000128 16737 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:38.000128 16743 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:18:38.003686 18972 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:38.004563 18972 consensus_queue.cc:1243] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:38.006239 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:38.028805 16741 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:18:38.018198 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.033064 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.040854 16737 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:18:38.062999 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:38.065979 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.029552 19070 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:18:38.071367 16745 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:18:38.079926 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48228.3us
I20260709 13:18:38.040321 18826 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:18:38.038703 19143 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:18:38.087091 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:38.088568 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53630.8us
I20260709 13:18:38.093926 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.095188 16737 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:38.105006 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.102065 19237 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:38.106076 19236 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:38.106763 19236 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:38.107998 19236 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.111169 19207 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:38.112372 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 167360us
I20260709 13:18:38.099524 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:38.111845 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.111812 19207 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:38.137640 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:38.143787 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:38.144565 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.133616 19218 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:38.162036 17409 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:38.162855 18996 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.164552 18996 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.167393 17412 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:38.168421 17420 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:38.169371 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:38.170253 19046 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.171377 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:38.172269 18934 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:38.172165 19178 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:38.172782 18934 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.173223 18934 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:38.174253 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.174403 19099 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:38.177510 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.179459 19178 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.180192 19178 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.181221 19180 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:38.181720 19180 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.182097 19180 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.183136 19178 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:38.187636 19247 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:18:38.188751 19207 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:38.190965 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.191475 19067 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:38.192009 19067 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:38.193097 19067 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.199410 16736 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:18:38.200248 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.211687 19070 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:38.212626 18742 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.214069 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.215242 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.216638 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.217312 16592 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:38.218240 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.219023 16594 raft_consensus.cc:2376] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8. Re-sending same reply.
I20260709 13:18:38.220664 16594 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:38.221923 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.156781 19218 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:38.223604 19044 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:38.224162 19044 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:38.224247 19218 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.225273 19044 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.225931 19218 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.227319 16494 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:38.228725 19253 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:38.228943 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.230674 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.231519 16587 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:38.235526 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.236491 16590 raft_consensus.cc:2376] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5. Re-sending same reply.
W20260709 13:18:38.237151 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.237327 19248 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:18:38.237761 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.222963 16595 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:38.238811 16596 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:38.229434 16494 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:18:38.240301 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.241194 16588 raft_consensus.cc:2410] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:38.242277 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.245347 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:38.246198 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.226680 19044 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.181453 19083 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:38.254189 19083 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:38.257956 19083 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.259369 19083 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:38.272542 16587 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:38.239996 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:38.278678 16592 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:38.262461 19232 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:38.304790 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.311916 18934 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.313342 19180 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.314532 16744 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:38.316748 19232 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:38.284622 18908 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
W20260709 13:18:38.319576 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.306396 18826 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:38.304905 19099 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:38.355260 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 174534us
W20260709 13:18:38.356945 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 123299us
I20260709 13:18:38.340435 18742 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:38.340123 18826 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:38.361099 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.367671 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.371374 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 245105us
I20260709 13:18:38.379889 18934 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.380648 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.359416 19099 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.383847 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:38.386533 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.370573 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 271486us
I20260709 13:18:38.363610 18826 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:38.403949 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.404495 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.410465 16742 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:18:38.405310 16741 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.415608 19207 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:38.412947 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.419813 19207 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:38.422129 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.424674 18972 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:38.425336 18972 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:38.427626 18951 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.427758 19178 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.428383 18951 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.428464 19178 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.427794 16592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:18:38.428416 16665 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:38.426680 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.420454 19207 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:38.416029 19180 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.440896 16741 raft_consensus.cc:2410] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:38.430284 18934 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.443751 19046 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.444969 19099 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.445569 19221 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:38.446043 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:38.446956 16670 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:18:38.453146 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56946.3us
W20260709 13:18:38.458597 19262 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:18:38.459578 19147 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:18:38.465220 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.467628 16741 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:38.464085 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.437995 19207 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.426400 18972 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.481364 19180 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.486456 19180 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.488734 19180 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.468698 19046 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.473134 18934 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.509106 16708 leader_election.cc:400] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
I20260709 13:18:38.524292 16708 leader_election.cc:403] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:18:38.516930 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.521528 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 99889.2us
W20260709 13:18:38.521761 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 68614.5us
I20260709 13:18:38.522212 19278 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:38.533787 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.534428 16592 raft_consensus.cc:606] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.534948 16592 raft_consensus.cc:614] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:38.565378 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:38.577171 19269 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:18:38.542456 18908 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:18:38.564851 19253 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:38.591990 19178 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:38.593511 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.593986 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.594013 16736 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:18:38.596037 19264 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:18:38.539302 19233 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
W20260709 13:18:38.601317 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 137442us
W20260709 13:18:38.602722 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 120900us
I20260709 13:18:38.560416 19268 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:38.615612 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75620.4us
I20260709 13:18:38.563283 19180 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:18:38.623590 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.624650 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:38.635869 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.615928 19253 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:38.638892 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.541433 19278 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
W20260709 13:18:38.640894 17374 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.606s (SENT)
W20260709 13:18:38.644343 19272 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:18:38.644513 19260 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:38.644999 19221 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:38.645455 19221 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:38.613855 19236 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:18:38.651051 16736 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:18:38.660138 16736 pending_rounds.cc:85] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 2
I20260709 13:18:38.660032 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.660485 16736 pending_rounds.cc:107] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:38.638684 18951 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.660750 16736 raft_consensus.cc:2889] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:38.655474 16741 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:38.663565 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:38.664149 16738 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:38.653465 19279 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.666226 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:38.662981 19233 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:38.668684 16744 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:38.649405 19083 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:38.669186 16743 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:38.638185 19276 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 13:18:38.643481 18908 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:18:38.646378 19221 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.649700 19260 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:38.676666 18972 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:38.680732 19259 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 13:18:38.681720 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68239us
I20260709 13:18:38.683554 19281 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:38.684132 19253 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.673352 19083 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:38.703951 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:38.703979 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:38.712595 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:38.720114 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 81451.2us
I20260709 13:18:38.723291 19260 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.725173 19286 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:18:38.732410 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:38.734946 19147 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:18:38.735862 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.643323 19268 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:38.712630 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.698530 18972 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:38.712428 19083 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.749714 16664 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:38.722496 18951 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.760414 16738 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:38.676558 19233 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:38.673190 19279 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:38.763638 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 104845us
I20260709 13:18:38.741637 18972 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.767519 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:38.755584 19221 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:38.770903 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66975us
I20260709 13:18:38.771453 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:38.721004 18826 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.776547 17419 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:18:38.784147 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:38.772110 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:38.791535 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:38.792135 16739 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:38.795081 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:38.795848 19261 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:38.777953 18934 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.799244 18934 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:38.802973 18826 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:38.803681 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:38.804287 16667 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:38.812176 17420 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:38.816042 16592 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:38.816792 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.817451 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.817730 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.818545 16595 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:38.819284 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.819643 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.820215 16589 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:38.820258 16595 raft_consensus.cc:2376] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4. Re-sending same reply.
I20260709 13:18:38.822808 16589 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:38.823597 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.834461 16589 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:38.792788 19180 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.773777 18951 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:38.841502 16593 raft_consensus.cc:2376] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7. Re-sending same reply.
W20260709 13:18:38.843339 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.843539 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.849251 16588 raft_consensus.cc:2376] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4. Re-sending same reply.
I20260709 13:18:38.825686 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:38.810950 17412 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.846517 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.855988 16594 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:18:38.857837 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:38.846932 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.852766 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.859623 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:38.860435 16587 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:38.861320 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:38.783912 19207 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:38.839710 19236 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.878011 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:38.989992 19099 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:38.932439 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52239us
I20260709 13:18:38.873183 16589 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:38.990273 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.777s (SENT)
I20260709 13:18:38.989563 19178 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:38.908733 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:38.996162 19266 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:38.875079 19278 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 13:18:38.874624 19288 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:38.997414 16587 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
W20260709 13:18:39.046571 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 205967us
W20260709 13:18:38.932394 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:38.941906 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.003080 19233 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:39.063594 16588 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:39.019037 19236 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.021590 19207 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:38.877641 19286 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:38.908152 19180 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:38.873229 19173 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:38.931591 19277 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:18:38.930763 19289 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:18:38.918706 18996 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.091125 19180 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:38.907457 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.091917 16662 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:39.048281 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.061254 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.062139 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.048466 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.060369 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.069185 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:39.076345 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 198401us
I20260709 13:18:39.037238 19254 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.099531 16665 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:39.093679 16666 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:39.079656 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:39.065492 16708 connection.cc:256] RPC call timeout handler was delayed by -1.8181318670001474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
I20260709 13:18:39.074026 18934 raft_consensus.cc:1064] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:39.046846 19288 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:39.089572 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:39.107527 16663 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.090283 19298 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:18:39.086390 19173 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.102996 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:39.104707 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69006.6us
I20260709 13:18:39.106477 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:39.037900 19099 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.047127 19178 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.119319 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.122166 16741 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.101255 19297 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.124940 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.110940 19286 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:39.107873 19288 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.132284 19083 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.110499 19176 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.135490 18951 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.110634 19180 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:18:39.136489 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:39.115739 19207 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.136945 19279 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:39.136948 16708 leader_election.cc:336] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.501s (SENT)
I20260709 13:18:39.141613 16742 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:39.135196 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.113572 19173 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.156780 16596 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:39.157940 16663 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:39.104046 19254 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.111866 19289 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:18:39.180851 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.181087 16669 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:39.181453 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 71755.8us
I20260709 13:18:39.124516 17412 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:39.185608 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.190114 16741 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:39.191602 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 77445.7us
I20260709 13:18:39.198025 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.182288 19221 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.179376 19279 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.201004 19284 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:39.201901 19286 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.162973 19266 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
W20260709 13:18:39.204442 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 106697us
I20260709 13:18:39.164391 18742 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:18:39.205703 16663 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:18:39.205231 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.205824 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.207224 18742 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:39.207542 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.207811 18742 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.208191 16671 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:39.208642 18742 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.209824 18742 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:39.207162 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90315.3us
I20260709 13:18:39.212306 18996 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.212862 18996 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.215312 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:39.209004 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:39.216028 19178 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.216594 19178 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:39.216845 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.218405 16671 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:18:39.216950 19178 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.194795 19252 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:39.223392 19254 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:39.164213 19176 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.238240 19143 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:18:39.238689 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74708.6us
W20260709 13:18:39.262436 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 272185us
I20260709 13:18:39.249024 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:39.224131 19276 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.254158 19308 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:39.254252 19286 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.262385 19268 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:39.188375 19292 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:18:39.233440 19178 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.289806 16665 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:39.206406 19207 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.274941 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.298121 19083 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.307857 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:39.300657 18934 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:39.303135 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 97986.9us
I20260709 13:18:39.306975 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.309988 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.279353 19281 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:18:39.328764 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 117540us
I20260709 13:18:39.277536 19143 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.312682 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:39.337577 16663 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:39.283484 19221 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:39.335837 17415 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:39.246753 18972 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:39.344691 16708 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.516s (SENT)
I20260709 13:18:39.343587 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:39.345335 17420 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:39.306214 19099 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.341948 19147 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:39.348140 19306 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.348446 19147 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.339181 19307 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.349438 19307 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.335388 19252 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.332998 19281 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.247856 19279 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:39.350492 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:39.356850 16668 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
W20260709 13:18:39.357757 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 110380us
I20260709 13:18:39.359324 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.312949 19288 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.333428 19304 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:18:39.354501 19281 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:39.355943 19083 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:39.353027 19236 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.359068 19180 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.369272 19288 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:39.366492 19254 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:39.370318 19306 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.374820 19306 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.354328 19252 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.370765 19279 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.371255 19067 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.376281 19067 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.377015 19252 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:39.380440 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6984318670001812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:18:39.380806 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6984318670001812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 13:18:39.381067 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6644065420000516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27320000000000005s for a response
I20260709 13:18:39.379385 19221 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.382542 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.385255 19147 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:39.385586 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48623us
I20260709 13:18:39.386994 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.377774 19067 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:39.450542 16708 leader_election.cc:336] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.206s (SENT)
I20260709 13:18:39.451104 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:18:39.451341 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64412.7us
I20260709 13:18:39.452474 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:39.452839 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64862.5us
I20260709 13:18:39.380339 19083 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.460243 19221 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.460765 19176 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.461292 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:39.462513 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 96102.7us
I20260709 13:18:39.474048 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.474946 16736 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:39.475024 18742 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:39.475483 18742 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.475813 18742 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.475983 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.476733 16740 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:39.476930 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:39.468357 19067 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.478969 19221 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.479241 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.479528 19221 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.480455 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:39.481786 19221 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.483306 18972 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:39.380007 19143 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.484164 19304 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:39.484400 19143 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:39.484701 19304 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.485739 19221 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.486270 19221 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.454347 19287 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:39.454147 19279 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:39.487826 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:39.489285 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.490958 16739 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:39.487026 19313 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.483338 19083 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.457360 19298 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:18:39.493350 16741 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:39.493562 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.379040 19207 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.491559 18996 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.497047 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.498973 16738 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.499187 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.495586 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:39.497882 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47424.2us
I20260709 13:18:39.498128 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.498267 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.500633 16739 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:39.499778 16745 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:18:39.501714 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.504716 19310 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.505263 19310 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:39.506454 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.507176 16745 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:39.507472 19310 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:39.507934 16708 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.389s (SENT)
I20260709 13:18:39.508556 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.486785 19304 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:39.513876 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.511039 16738 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:39.511018 16739 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:39.497109 19147 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.515584 16596 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 7. Ops: []
I20260709 13:18:39.494580 19307 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:39.516050 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.523707 16590 raft_consensus.cc:2376] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6. Re-sending same reply.
W20260709 13:18:39.518357 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.522606 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.522708 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.525705 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:39.526153 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.526919 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.527458 16595 raft_consensus.cc:2376] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6. Re-sending same reply.
I20260709 13:18:39.524726 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.528877 16590 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:39.529570 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.530210 16590 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:39.530210 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.531064 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.531458 16739 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:39.532004 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.532743 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.532790 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.533578 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.508497 18972 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.534760 16593 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:39.515074 19279 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.535753 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.496410 19099 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:18:39.536871 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.541182 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.541473 16596 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:39.541572 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.538166 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.539744 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:39.523682 19304 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:39.543671 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.545778 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.546547 16595 raft_consensus.cc:2410] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:39.546700 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:39.547212 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.547833 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:39.548312 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44556.8us
I20260709 13:18:39.547765 16743 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:39.534750 18951 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:39.549490 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:39.527982 19311 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.503222 19313 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.551915 16664 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.539239 19067 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.548993 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:39.553772 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60248.9us
I20260709 13:18:39.528733 19147 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.562945 18742 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.536774 19046 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.555608 16664 raft_consensus.cc:614] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:39.555792 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.564947 18742 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:39.557324 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.565552 16590 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:39.565717 16667 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:39.554471 19297 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:39.566195 16708 leader_election.cc:336] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.451s (SENT)
I20260709 13:18:39.566634 19297 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:18:39.567014 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6192318670002805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1824s for a response
I20260709 13:18:39.567039 19297 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:39.567432 16708 connection.cc:256] RPC call timeout handler was delayed by -1.6147318670002733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 13:18:39.567595 19297 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:39.567785 16708 connection.cc:256] RPC call timeout handler was delayed by -1.5544318670001758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
W20260709 13:18:39.568231 16708 connection.cc:256] RPC call timeout handler was delayed by -1.4302318670002023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 13:18:39.568516 16708 connection.cc:256] RPC call timeout handler was delayed by -1.262831867000159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 13:18:39.568770 16708 connection.cc:256] RPC call timeout handler was delayed by -1.1368318670001827s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.236s for a response
W20260709 13:18:39.569010 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0441318670002602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24630000000000002s for a response
I20260709 13:18:39.569774 19314 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:39.560246 16595 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:39.560336 19311 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.561383 16738 raft_consensus.cc:2410] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:18:39.562482 19313 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.563135 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.554842 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.557518 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.575621 16670 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:39.573235 16592 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:39.565472 16740 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:39.543179 18972 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.564972 19147 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:39.540326 19310 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:18:39.579905 18996 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:39.580427 19173 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.587409 19221 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:39.588366 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.603s (SENT)
I20260709 13:18:39.589823 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.551414 19254 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.551407 19180 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.611908 19180 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.612949 19180 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.551558 19298 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.549299 19176 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.615092 19298 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:39.594509 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45583.5us
I20260709 13:18:39.595389 16591 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:39.579825 16741 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:39.617342 19298 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:39.603693 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44936.5us
I20260709 13:18:39.617735 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.619050 19176 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.620330 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:39.610502 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:39.622073 16708 leader_election.cc:336] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.633s (SENT)
I20260709 13:18:39.596871 19311 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.623883 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.624387 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:39.543053 19279 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.626302 16591 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
W20260709 13:18:39.626852 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.627492 19099 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:39.631317 19176 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.631919 19176 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:39.632304 19176 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.632819 16744 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:39.632951 19173 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.633639 18742 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:39.634516 18742 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:39.634920 18742 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.635877 19268 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:39.635818 18742 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.637127 18742 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.640890 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.641587 16595 raft_consensus.cc:2376] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6. Re-sending same reply.
I20260709 13:18:39.627635 19067 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:39.642591 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.644913 16595 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:39.646190 16741 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:18:39.646581 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.653142 16592 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:18:39.654959 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.646546 16741 pending_rounds.cc:85] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 3
I20260709 13:18:39.664013 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:39.664706 16708 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.736s (SENT)
I20260709 13:18:39.665096 16595 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:39.665189 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.664281 16741 pending_rounds.cc:107] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:39.666273 16741 raft_consensus.cc:2889] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:39.666492 16592 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:39.667131 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.667634 16740 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:39.670992 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:39.671571 16662 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:39.672116 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.645465 19284 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.677201 16595 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:39.678412 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.679091 16595 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:39.680732 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56901.6us
I20260709 13:18:39.669538 19147 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:18:39.681751 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.674728 18996 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.623420 19254 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:39.696333 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 106570us
I20260709 13:18:39.712961 19254 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:39.696925 16741 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [5.2-7.3]
I20260709 13:18:39.684582 19284 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:39.699209 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50990.3us
I20260709 13:18:39.682969 19326 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.687942 19313 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.703049 18972 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:39.674265 19253 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:18:39.706590 19279 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.712837 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:39.652419 19281 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.697679 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:39.727258 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.748418 16595 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:39.735577 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.736s (SENT) [suppressed 38 similar messages]
I20260709 13:18:39.753247 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.754712 16741 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:39.755174 16740 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:39.756845 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.760673 16594 raft_consensus.cc:2368] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11.
I20260709 13:18:39.760072 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.757627 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:39.759087 19304 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:39.759297 19306 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.765549 19306 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.757244 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.733767 19297 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.776937 19297 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.778357 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.778098 19298 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:39.779469 16592 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:39.780790 19298 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.781948 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.782526 16592 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:39.783077 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.783882 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:39.783118 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47576.6us
I20260709 13:18:39.784584 16592 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:39.785138 16740 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:39.786888 16589 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:39.785471 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.697494 19178 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.726276 19173 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:18:39.793306 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.794450 16595 raft_consensus.cc:2376] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:39.796182 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.796890 16595 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:39.806084 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:39.806887 16587 raft_consensus.cc:2368] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
I20260709 13:18:39.794242 18742 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:39.807803 18742 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.808960 16595 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:18:39.727823 19326 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:39.812270 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 126840us
I20260709 13:18:39.812647 18742 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.751251 19279 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:18:39.815296 19298 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:39.814219 19253 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:39.815722 19298 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:39.816112 19253 consensus_queue.cc:1243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:39.816715 19298 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.817579 19180 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.817902 19298 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:39.818797 19143 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:39.819242 19143 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.820118 19143 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.821074 17374 leader_election.cc:400] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:39.821393 17374 leader_election.cc:403] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:18:39.822180 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.822669 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:39.824108 19046 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:39.824739 19046 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:39.825073 19046 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:39.825412 19046 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:39.825551 17412 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:39.828827 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 234) took 1025 ms. Trace:
I20260709 13:18:39.829214 16668 rpcz_store.cc:276] 0709 13:18:38.803323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.846111 (+ 42788us) service_pool.cc:224] Handling call
0709 13:18:39.828633 (+982522us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b540009ef54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:39.828740 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741704576}
I20260709 13:18:39.831064 19212 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:18:39.831435 19308 raft_consensus.cc:670] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } [suppressed 5 similar messages]
I20260709 13:18:39.832834 19212 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:39.833743 19083 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:39.834110 19083 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:39.834897 19083 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:39.898818 16708 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.829s (SENT)
I20260709 13:18:39.799902 19260 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:39.902442 19176 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:39.906279 18996 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.906724 19254 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:39.907920 19281 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.913630 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 200) took 1092 ms. Trace:
I20260709 13:18:39.934707 16588 rpcz_store.cc:276] 0709 13:18:38.821411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.863538 (+ 42127us) service_pool.cc:224] Handling call
0709 13:18:39.048445 (+184907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.064173 (+ 15728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.913433 (+849260us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54000d98d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:39.913567 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424485248}
I20260709 13:18:39.914242 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.809875 19284 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.926697 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:39.923821 19334 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.116s	user 0.000s	sys 0.002s
W20260709 13:18:39.924088 19327 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.112s	user 0.003s	sys 0.000s
I20260709 13:18:39.940186 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:39.950073 16742 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:39.939841 16589 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:18:39.943439 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:39.951798 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.944651 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:39.952978 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 198) took 1132 ms. Trace:
I20260709 13:18:39.954221 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.954756 16738 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:39.953596 16589 rpcz_store.cc:276] 0709 13:18:38.820580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.822286 (+  1706us) service_pool.cc:224] Handling call
0709 13:18:38.822640 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:39.786412 (+963772us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54002a6fa0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb59a1d 00007f327fb401a4 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419
0709 13:18:39.952689 (+166277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:39.952917 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308663040}
I20260709 13:18:39.956367 16589 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:39.958796 16589 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:39.960057 19178 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.975162 16588 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:18:39.944903 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:39.983584 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.984035 18908 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:39.984380 16662 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:39.984664 18908 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:39.985222 19176 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:39.947106 19252 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:39.983666 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:39.970566 19253 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:39.938207 18996 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.025828 16744 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:39.992265 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 180476us
I20260709 13:18:39.939496 19254 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:40.027347 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:39.944125 19279 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:18:39.954139 19260 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:40.024842 16738 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:18:40.033033 16740 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:40.053539 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.054039 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.054459 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:39.986833 18908 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:40.064183 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.069455 16670 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:40.066162 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.064354 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:40.068579 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79866.8us
I20260709 13:18:40.072701 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:40.075737 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:39.989725 19316 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:18:40.077138 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:40.077298 16708 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.896s (SENT)
I20260709 13:18:40.077888 16669 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:40.078384 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.028570 19178 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.080363 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:40.036350 19253 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:40.083864 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:40.094660 16740 raft_consensus.cc:2368] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:40.087555 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:40.089440 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.099135 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:40.102510 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.068323 19310 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.108749 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.110088 16589 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:40.034364 19288 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:18:40.112542 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.116187 16744 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:40.119728 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:40.120442 16744 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.121714 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:40.122282 16742 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.115343 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:40.125272 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:40.126917 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:18:40.130374 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:40.122933 19278 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.132082 19278 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.134298 18996 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:40.135515 19332 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:18:40.135948 19326 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:40.092242 19176 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.138206 19254 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:40.154513 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85235.9us
I20260709 13:18:40.141275 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:40.098579 19253 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.104957 19173 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:40.145133 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.164341 16669 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.098094 19306 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:40.150014 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.156203 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:40.158033 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.106336 19304 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:40.162768 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.147467 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.225232 17420 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.134619 19278 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:40.209328 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.226343 17415 raft_consensus.cc:2410] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
W20260709 13:18:40.211831 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.213338 16744 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:40.123080 19313 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:40.223541 16669 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:40.224829 16742 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:40.167927 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:40.228868 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61068.2us
I20260709 13:18:40.230834 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:40.232678 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:40.227005 19304 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.111092 19178 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.159186 19221 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:40.224776 16739 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
W20260709 13:18:40.230772 19296 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:18:40.239590 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.236868 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:40.124006 19310 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:40.218264 19254 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:40.248288 16667 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:40.201840 19306 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.260115 19306 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.270664 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.271324 16671 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.272593 16742 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:40.273862 16667 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:18:40.163640 19176 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.201550 19046 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.237697 19313 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:18:40.292439 16744 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:18:40.319754 16708 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.732s (SENT)
I20260709 13:18:40.240482 19278 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:40.333093 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.240051 19304 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.296397 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.309911 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:40.259382 19254 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.319850 16667 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:40.320386 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:40.293718 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.258980 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:40.338167 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97130.3us
I20260709 13:18:40.399783 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:40.293975 19279 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:40.319162 17413 raft_consensus.cc:2368] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8.
I20260709 13:18:40.398478 16668 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:40.405723 16559 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.427659 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.293033 19306 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.398068 19254 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.331650 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:40.469120 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 258519us
I20260709 13:18:40.419121 19278 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:40.477433 19288 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:40.428395 16664 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
W20260709 13:18:40.478237 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 245632us
I20260709 13:18:40.512437 16596 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:18:40.506987 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 187261us
I20260709 13:18:40.425891 19313 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:40.423909 17409 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:40.542713 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.467646 17410 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:40.507090 19067 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.558893 19313 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:40.583350 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.558207 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:40.559075 19356 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.560163 19330 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:18:40.575182 19284 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:40.672613 17411 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:40.662181 16668 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:40.672616 19356 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.663327 19364 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.669031 16589 pending_rounds.cc:85] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 3
I20260709 13:18:40.675381 16589 pending_rounds.cc:107] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:40.429198 19351 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.145s
I20260709 13:18:40.674460 19364 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.675800 16589 raft_consensus.cc:2889] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:40.675365 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 165879us
I20260709 13:18:40.660609 19370 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:40.677445 17376 leader_election.cc:400] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:40.677521 19364 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:40.677793 17376 leader_election.cc:403] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:18:40.677906 19370 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.678830 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:40.678589 19370 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.681005 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:40.681470 19370 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:40.682305 19308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.001s	sys 0.008s
W20260709 13:18:40.682797 19308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.001s	sys 0.008s
W20260709 13:18:40.683707 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.682232 19370 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:40.683533 16595 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.001s	sys 0.009s
W20260709 13:18:40.711828 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:40.712345 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 306711us
W20260709 13:18:40.726781 16595 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.001s	sys 0.012s
W20260709 13:18:40.684072 19327 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:18:40.751113 19370 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:40.769737 19376 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:40.770687 19376 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:40.771214 19376 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.772760 19370 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.773576 19376 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:40.774031 19379 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:40.775902 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.684865 19173 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:18:40.670929 18972 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 13:18:40.750224 17414 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:40.795569 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:40.770707 19306 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:40.798171 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 391413us
I20260709 13:18:40.792013 19284 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:40.810271 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:40.811137 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 268571us
I20260709 13:18:40.849781 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:40.800580 19353 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:18:40.799858 19365 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:40.798492 19178 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:40.806988 19341 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.420s	user 0.006s	sys 0.000s
I20260709 13:18:40.855453 19358 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:40.857412 19375 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:18:40.805303 19373 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:40.810992 19252 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.808468 19351 consensus_queue.cc:1243] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:40.811679 19379 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:40.874243 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66884.4us
I20260709 13:18:40.804701 19306 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.849237 17412 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:18:40.847745 19284 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:40.849665 19046 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:40.809046 19310 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:40.849974 19176 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:40.925284 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.925901 16668 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:40.926625 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:40.927263 16664 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:40.927464 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2384) took 1252 ms. Trace:
W20260709 13:18:40.877652 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 67440.4us
I20260709 13:18:40.796582 19278 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:40.913640 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 63944.8us
I20260709 13:18:40.916347 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:40.921437 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:40.932928 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:40.935063 19373 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:40.935448 19253 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.719s	user 0.008s	sys 0.002s
I20260709 13:18:40.935920 19253 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:40.938006 19253 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:40.939121 19254 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.941783 19254 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:40.944252 19304 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.543s	user 0.006s	sys 0.003s
I20260709 13:18:40.944661 19304 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.914381 19358 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.949604 16738 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:40.927807 16591 rpcz_store.cc:276] 0709 13:18:39.675184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.911664 (+236480us) service_pool.cc:224] Handling call
0709 13:18:39.912025 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.912376 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.912410 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.912865 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.005865 (+ 93000us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:40.162682 (+156817us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.294609 (+131927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.295826 (+  1217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.927280 (+631454us) raft_consensus.cc:1685] finished
0709 13:18:40.927300 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:40.927411 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58147,"prepare.queue_time_us":233540,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":78621,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":40042,"thread_start_us":323002,"threads_started":3,"wal-append.queue_time_us":210552}
I20260709 13:18:40.913985 19173 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:40.922361 19306 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.952242 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:40.952513 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:40.953070 16559 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.946611 19304 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:40.959044 17419 raft_consensus.cc:2410] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:40.964183 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:40.938017 19298 consensus_meta.cc:287] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.120s	user 0.011s	sys 0.000s
I20260709 13:18:40.984880 16559 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.986409 16559 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.990399 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:40.994400 19370 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:40.995193 19370 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:40.998777 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 163599us
I20260709 13:18:40.935485 19373 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:40.942744 17409 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:40.918498 19252 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:41.004293 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.011309 19389 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:41.011976 19388 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:41.017078 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54915.9us
I20260709 13:18:40.962951 19307 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:40.966284 19310 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:40.988929 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.025635 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:40.988845 19178 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:40.965286 19383 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:40.962091 19254 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:18:40.961565 19173 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:40.881834 19365 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.007055 19373 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.013409 19364 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:41.037349 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:41.016604 19389 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:40.921571 19379 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.051695 16559 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:41.034153 19176 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:41.055003 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68731.9us
I20260709 13:18:41.055823 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:41.057075 16671 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.762s	user 0.010s	sys 0.000s
I20260709 13:18:41.057047 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.055383 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:41.057617 16671 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:41.057801 16670 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:41.058697 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.059230 16668 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:41.060691 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:41.061211 16667 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.048888 19278 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:41.062440 16636 leader_election.cc:400] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8.
I20260709 13:18:41.064198 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:41.066183 19268 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:41.031847 19310 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.034351 19298 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:18:41.014238 19284 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.031594 19307 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.094112 19307 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.070000 19278 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:41.094625 19310 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.071913 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:41.071985 19390 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.072470 19386 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.072871 19401 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.073186 19384 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.070497 19393 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:41.076645 19405 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:41.075511 19394 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:41.054586 19389 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.122215 19401 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.079274 19400 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:41.126313 19401 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.080103 19370 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.081470 19379 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.030370 19388 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.063001 17414 raft_consensus.cc:2410] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:18:41.083096 19365 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.130956 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.083266 19395 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.132812 19395 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.133581 19384 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.134163 19384 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.137537 19356 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.035939 19178 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.060086 19176 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.138583 19176 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.139596 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1835) took 1626 ms. Trace:
I20260709 13:18:41.120700 19390 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.140028 16587 rpcz_store.cc:276] 0709 13:18:39.513513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.513636 (+   123us) service_pool.cc:224] Handling call
0709 13:18:41.139413 (+1625777us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540023bed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:41.139540 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656554880}
I20260709 13:18:41.141285 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.141623 19390 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.083582 19358 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.120652 19310 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.083947 19254 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:41.085577 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:41.143286 19310 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.143285 19254 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.144307 19254 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.145358 19310 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.087098 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2394) took 1391 ms. Trace:
I20260709 13:18:41.138952 19384 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.141824 16587 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:41.088152 19306 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.089359 19252 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.092309 19173 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.095320 16668 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.096300 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:41.090282 19373 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.097160 19143 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:41.148068 16745 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:41.149560 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:41.150398 19173 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:41.150465 19373 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.132253 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.066808 19268 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:18:41.145898 16596 rpcz_store.cc:276] 0709 13:18:39.695238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.663242 (+968004us) service_pool.cc:224] Handling call
0709 13:18:40.663647 (+   405us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:40.793069 (+129422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.793111 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:40.847078 (+ 53967us) log.cc:844] Serialized 78 byte log entry
0709 13:18:40.949542 (+102464us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:40.949581 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.949655 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.086889 (+137234us) raft_consensus.cc:1685] finished
0709 13:18:41.086908 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.087047 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48347,"raft.queue_time_us":127516,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":5436,"thread_start_us":135012,"threads_started":2,"wal-append.queue_time_us":180388}
I20260709 13:18:41.096987 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:41.100292 16667 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:41.148170 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.150842 19173 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:41.151786 19178 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.139608 19393 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.153468 19351 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.158339 16668 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:41.165464 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2398) took 1187 ms. Trace:
I20260709 13:18:41.162245 16587 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:41.164619 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:41.165835 16589 rpcz_store.cc:276] 0709 13:18:39.977699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.668136 (+690437us) service_pool.cc:224] Handling call
0709 13:18:40.668680 (+   544us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:40.887434 (+218754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.887479 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:18:40.889039 (+  1560us) log.cc:844] Serialized 600 byte log entry
0709 13:18:40.984074 (+ 95035us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:41.009438 (+ 25364us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.013601 (+  4163us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.164643 (+151042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.165199 (+   556us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.165261 (+    62us) raft_consensus.cc:1685] finished
0709 13:18:41.165276 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.165414 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56957,"prepare.queue_time_us":31188,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":100,"raft.queue_time_us":203639,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":2453,"spinlock_wait_cycles":9870208,"thread_start_us":368184,"threads_started":3,"wal-append.queue_time_us":77808}
I20260709 13:18:41.163026 19383 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:41.163101 19173 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.160997 19395 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.177564 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:41.179836 19396 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.185452 19396 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.186510 19393 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.181846 19387 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.172384 19351 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.188423 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:41.189365 19351 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.189889 19070 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:41.191090 19070 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.191468 19070 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.191702 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.192369 16588 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:41.193843 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2400) took 1215 ms. Trace:
I20260709 13:18:41.194193 16588 rpcz_store.cc:276] 0709 13:18:39.978166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.193011 (+1214845us) service_pool.cc:224] Handling call
0709 13:18:41.193339 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.193516 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.193542 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:41.193585 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:41.193619 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.193673 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.193790 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:41.181236 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:41.196645 16589 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.196163 16588 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:41.173362 19389 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.199399 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.203423 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2401) took 1219 ms. Trace:
I20260709 13:18:41.203927 16588 rpcz_store.cc:276] 0709 13:18:39.978387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.195412 (+1217025us) service_pool.cc:224] Handling call
0709 13:18:41.195840 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.197375 (+  1535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.197542 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:41.206677 16590 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1844) took 1688 ms (client timeout 2184 ms). Trace:
W20260709 13:18:41.207116 19349 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.803s	user 0.000s	sys 0.003s
W20260709 13:18:41.207268 16590 rpcz_store.cc:269] 0709 13:18:39.517932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.530854 (+ 12922us) service_pool.cc:224] Handling call
0709 13:18:41.206426 (+1675572us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b5400157f54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:41.206589 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78065536}
I20260709 13:18:41.209182 16587 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:41.209714 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 154948us
W20260709 13:18:41.210175 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 154855us
W20260709 13:18:41.228901 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.238955 16588 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:41.240012 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2403) took 1251 ms. Trace:
I20260709 13:18:41.240469 16588 rpcz_store.cc:276] 0709 13:18:39.989035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.204817 (+1215782us) service_pool.cc:224] Handling call
0709 13:18:41.238450 (+ 33633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.239695 (+  1245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.239942 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:41.223770 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:41.201067 19254 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.246040 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 205) took 1172 ms. Trace:
I20260709 13:18:41.246565 16587 rpcz_store.cc:276] 0709 13:18:40.073186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.209967 (+1136781us) service_pool.cc:224] Handling call
0709 13:18:41.245418 (+ 35451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.245639 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.245677 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:41.245730 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:41.245770 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.245844 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.245973 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:41.209656 19178 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.249320 19297 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:41.250049 19178 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.250669 19297 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:41.252960 16559 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:41.255935 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:41.260349 16589 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:41.263191 16708 leader_election.cc:400] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11.
I20260709 13:18:41.264068 16587 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:41.251191 19297 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:41.266584 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:41.272177 19358 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:41.272639 19358 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.273007 19358 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.274312 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.274561 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.274963 16670 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:41.275151 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.275399 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.275907 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.276435 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.263186 19373 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:41.275975 16668 raft_consensus.cc:2410] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:18:41.278265 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 212742us
I20260709 13:18:41.265959 19345 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:41.282603 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:41.266243 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.269649 19326 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:41.336572 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54062.8us
I20260709 13:18:41.336596 16667 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:41.337409 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 992) took 1546 ms. Trace:
W20260709 13:18:41.279223 19279 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.022s	sys 0.011s
I20260709 13:18:41.367807 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:41.368615 16741 rpcz_store.cc:276] 0709 13:18:39.736929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.770333 (+ 33404us) service_pool.cc:224] Handling call
0709 13:18:39.770732 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.772872 (+  2140us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:40.057815 (+284943us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.209642 (+151827us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:40.210098 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.210225 (+   127us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:40.238391 (+ 28166us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.259050 (+ 20659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.259124 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.206273 (+947149us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:41.282781 (+ 76508us) raft_consensus.cc:1685] finished
0709 13:18:41.282807 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.282943 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230661,"prepare.queue_time_us":676490,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":38748,"raft.queue_time_us":63584,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":38393,"spinlock_wait_cycles":299004928,"thread_start_us":177666,"threads_started":2,"wal-append.queue_time_us":192765}
I20260709 13:18:41.368856 16708 leader_election.cc:400] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
I20260709 13:18:41.283881 19345 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.284242 19373 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:41.376374 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:41.367624 19370 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:41.372242 19279 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.023s	sys 0.015s
I20260709 13:18:41.378854 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:41.381330 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:41.370620 19326 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.377264 19345 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.373708 17414 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:41.379957 19370 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.380607 19393 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:41.381268 19279 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:41.383248 19391 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:41.394104 19391 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:41.395147 19391 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.396404 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2399) took 1418 ms. Trace:
I20260709 13:18:41.396836 16591 rpcz_store.cc:276] 0709 13:18:39.977943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.950370 (+972427us) service_pool.cc:224] Handling call
0709 13:18:40.950687 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:40.986016 (+ 35329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.986043 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.000358 (+ 14315us) log.cc:844] Serialized 26 byte log entry
0709 13:18:41.161535 (+161177us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:41.161574 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.162227 (+   653us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.396188 (+233961us) raft_consensus.cc:1685] finished
0709 13:18:41.396210 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.396342 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34918,"raft.queue_time_us":56568,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9608,"thread_start_us":148596,"threads_started":1,"wal-append.queue_time_us":253517}
I20260709 13:18:41.398761 19393 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:41.400528 19070 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.401930 16741 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:41.402570 19070 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.402745 19394 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:41.404768 19394 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:41.407018 19343 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.986s	user 0.003s	sys 0.008s
I20260709 13:18:41.409284 16591 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:41.410307 19394 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.392808 19389 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.385784 19356 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:41.372601 19297 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.387218 19326 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:41.426563 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45286us
I20260709 13:18:41.393054 19370 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.434401 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:41.435992 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:41.420313 19252 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:41.430014 19356 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:41.434204 19410 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:18:41.399744 19393 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.439545 19143 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:41.439584 19384 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.438004 19400 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:41.441313 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:41.434000 19254 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.449518 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.442023 19279 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.439138 19358 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.457006 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2381) took 1786 ms. Trace:
I20260709 13:18:41.458006 19147 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:41.458437 19147 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:41.459347 19147 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.448514 19356 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:41.460918 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2406) took 1134 ms. Trace:
I20260709 13:18:41.452493 17414 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:41.462271 16589 rpcz_store.cc:276] 0709 13:18:40.326721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.261914 (+935193us) service_pool.cc:224] Handling call
0709 13:18:41.262347 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.271991 (+  9644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.272064 (+    73us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.273001 (+   937us) log.cc:844] Serialized 26 byte log entry
0709 13:18:41.381894 (+108893us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:41.381919 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.381962 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.460690 (+ 78728us) raft_consensus.cc:1685] finished
0709 13:18:41.460712 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.460858 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3921,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"thread_start_us":108413,"threads_started":1,"wal-append.queue_time_us":165529}
I20260709 13:18:41.457377 16595 rpcz_store.cc:276] 0709 13:18:39.671013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.869187 (+198174us) service_pool.cc:224] Handling call
0709 13:18:39.901515 (+ 32328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.927430 (+ 25915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.927456 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.937528 (+ 10072us) log.cc:844] Serialized 26 byte log entry
0709 13:18:40.076324 (+138796us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:40.727369 (+651045us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.795872 (+ 68503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.795958 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.247911 (+451953us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:41.452843 (+204932us) raft_consensus.cc:1685] finished
0709 13:18:41.456801 (+  3958us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.456945 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61739,"prepare.queue_time_us":214320,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":154649,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":44921,"spinlock_wait_cycles":333143296,"thread_start_us":803935,"threads_started":3,"wal-append.queue_time_us":279838}
I20260709 13:18:41.455888 19252 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:41.452935 19176 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.432425 19373 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.466861 19260 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:41.467324 19260 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:41.468361 19252 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:41.469201 16588 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:41.469007 19252 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.453761 19143 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.468472 19176 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.468161 19260 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.472121 19260 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:41.471840 19143 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.476720 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:41.486541 17414 raft_consensus.cc:2376] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7. Re-sending same reply.
I20260709 13:18:41.487701 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.488476 17414 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:41.489351 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.489903 17414 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 3.
I20260709 13:18:41.478711 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.482646 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.483392 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.493013 16663 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:41.483852 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:41.493839 19364 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:41.494211 16669 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:41.483754 16662 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:41.495224 16662 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:41.497534 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:41.485605 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 452) took 1322 ms. Trace:
I20260709 13:18:41.461483 19391 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:41.499414 19306 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:41.500072 16662 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.509474 16669 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:41.454195 19173 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.494665 19252 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.503751 19306 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.516754 19356 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.494393 19364 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:41.524034 19221 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.004s	sys 0.013s
I20260709 13:18:41.501273 17410 rpcz_store.cc:276] 0709 13:18:40.163537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.218978 (+ 55441us) service_pool.cc:224] Handling call
0709 13:18:40.293904 (+ 74926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:40.670310 (+376406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.485317 (+815007us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54005e1d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:41.485516 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259581824}
I20260709 13:18:41.524185 19306 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:41.532159 17411 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:41.530355 19221 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.260s	user 0.005s	sys 0.013s
I20260709 13:18:41.533167 19221 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:41.534175 19221 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.535187 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2392) took 1856 ms. Trace:
I20260709 13:18:41.535593 16594 rpcz_store.cc:276] 0709 13:18:39.679253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.319139 (+639886us) service_pool.cc:224] Handling call
0709 13:18:40.399119 (+ 79980us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:40.684615 (+285496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.711275 (+ 26660us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:40.740960 (+ 29685us) log.cc:844] Serialized 52 byte log entry
0709 13:18:41.196423 (+455463us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b1800396d60. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:18:41.368328 (+171905us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:41.377308 (+  8980us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.384247 (+  6939us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.390853 (+  6606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.390970 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.486653 (+ 95683us) spinlock_profiling.cc:243] Waited 1.26 ms on lock 0x7b4800002958. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:18:41.488362 (+  1709us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b1800396d60. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:41.534926 (+ 46564us) raft_consensus.cc:1685] finished
0709 13:18:41.534948 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.535114 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32117,"prepare.queue_time_us":30884,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":327093,"raft.run_cpu_time_us":1393,"raft.run_wall_time_us":139935,"spinlock_wait_cycles":103557248,"thread_start_us":435541,"threads_started":2,"wal-append.queue_time_us":619077}
I20260709 13:18:41.512090 19391 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.540910 19358 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.541505 16594 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:18:41.541997 16594 pending_rounds.cc:85] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 6
I20260709 13:18:41.542279 16594 pending_rounds.cc:107] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:18:41.542718 16594 raft_consensus.cc:2889] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:41.527104 19396 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.550060 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.539031 19252 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:41.555869 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 103360us
W20260709 13:18:41.547266 19325 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.255s	user 0.017s	sys 0.008s
I20260709 13:18:41.560417 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:41.560959 19427 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:18:41.556838 19252 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.564376 19408 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:41.566138 19252 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.556047 17409 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:41.557793 19370 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.572733 19173 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.573450 19173 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.573923 19173 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.583062 17414 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:41.583313 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:41.583478 19306 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.584746 19408 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.585716 19306 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.586846 19400 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.587373 19400 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.589454 19400 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.590070 19400 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.590502 19400 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.574712 19370 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.591502 19356 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.584849 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 449) took 1475 ms. Trace:
I20260709 13:18:41.592065 17420 rpcz_store.cc:276] 0709 13:18:40.109124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.121800 (+ 12676us) service_pool.cc:224] Handling call
0709 13:18:41.584666 (+1462866us) spinlock_profiling.cc:243] Waited 94 ms on lock 0x7b54004e5554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:41.584769 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216114944}
I20260709 13:18:41.592489 19400 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:41.592844 19355 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.098s	user 0.007s	sys 0.004s
I20260709 13:18:41.593567 19400 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.593709 17420 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:41.594084 19400 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:41.594592 19342 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.290s	user 0.005s	sys 0.005s
I20260709 13:18:41.585382 19408 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.595484 17420 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:41.595527 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:41.568459 19325 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.018s	sys 0.008s
I20260709 13:18:41.569414 19345 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.619553 19364 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.617084 19252 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.621161 19252 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.625777 19252 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.626025 19408 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.626507 19350 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:41.633005 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.633919 17419 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:41.611747 19326 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.643643 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:41.626994 19350 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:41.654446 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.647475 19408 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:41.659134 19400 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.659449 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:41.657702 19408 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.661378 19408 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:41.661494 19326 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.661609 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 447) took 1553 ms. Trace:
I20260709 13:18:41.664692 17415 rpcz_store.cc:276] 0709 13:18:40.108549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.122046 (+ 13497us) service_pool.cc:224] Handling call
0709 13:18:41.661420 (+1539374us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:41.661554 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229979008}
I20260709 13:18:41.667548 19400 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.668093 19400 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.670305 19400 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.663074 19391 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.664404 17410 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:41.682983 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:41.684955 16664 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.662132 19350 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.683720 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:41.689723 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:41.691992 18742 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:41.692366 18742 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.692660 18742 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:41.698174 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:41.701205 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:41.695006 19431 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:18:41.710809 18908 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:41.711455 18908 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:41.697661 19393 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:41.712002 19393 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.711810 19409 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:41.712409 18908 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.714049 19409 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:41.702390 19350 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.715114 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:41.716733 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:41.718287 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:41.718847 17415 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:41.719977 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.720741 16669 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:41.712404 19393 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:41.729753 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.710371 19173 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.704092 19253 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:18:41.718006 19260 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:41.735714 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:41.728936 19143 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.749343 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 206) took 1230 ms. Trace:
I20260709 13:18:41.750324 16634 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:41.751386 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:41.730728 19345 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.761284 16664 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:41.758566 17415 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:41.754848 18742 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:41.759487 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.749768 16587 rpcz_store.cc:276] 0709 13:18:40.518808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.268725 (+749917us) service_pool.cc:224] Handling call
0709 13:18:41.282172 (+ 13447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.374610 (+ 92438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.374630 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.377733 (+  3103us) log.cc:844] Serialized 26 byte log entry
0709 13:18:41.431867 (+ 54134us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:41.437370 (+  5503us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.749070 (+311700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.749136 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:41.749173 (+    37us) raft_consensus.cc:1685] finished
0709 13:18:41.749186 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.749291 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":412774,"prepare.queue_time_us":3790,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":408640,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":5622,"thread_start_us":43479,"threads_started":1,"wal-append.queue_time_us":73761}
I20260709 13:18:41.768565 16663 raft_consensus.cc:2368] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:41.786135 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.780005 19351 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:41.790210 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:41.790719 18742 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:41.779764 19310 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.780824 19381 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:41.797389 17420 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 5 LEARNER]: Advancing to term 7
I20260709 13:18:41.788167 19260 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:41.799983 17412 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:41.790745 19351 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:41.779285 19143 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.783408 19286 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:18:41.792759 19358 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.795055 19298 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.804932 19345 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.830576 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.806680 19405 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:41.808070 19351 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:41.834254 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.807766 19260 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:41.816958 19412 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:41.830145 19345 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.840867 17420 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 7 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:41.811338 19310 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.844734 19412 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.851670 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.852356 16594 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:41.853214 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.853936 19412 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.854207 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:41.854759 16596 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:41.868106 19143 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.873524 19351 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.829344 19298 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.876525 19345 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:41.876629 19298 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.880769 19310 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.881521 19260 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.826936 19358 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.887492 16594 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:41.881035 16737 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:41.839769 19405 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:41.904174 19284 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:41.904510 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1848) took 2385 ms. Trace:
I20260709 13:18:41.905035 19358 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:41.908102 19178 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:41.913292 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:41.964536 16593 rpcz_store.cc:276] 0709 13:18:39.518536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.565028 (+ 46492us) service_pool.cc:224] Handling call
0709 13:18:39.647138 (+ 82110us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:39.669369 (+ 22231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:39.669401 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:39.697427 (+ 28026us) log.cc:844] Serialized 26 byte log entry
0709 13:18:39.788491 (+ 91064us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:40.039918 (+251427us) log.cc:844] Serialized 14 byte log entry
0709 13:18:40.040200 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:40.040294 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:40.106036 (+ 65742us) raft_consensus.cc:1685] finished
0709 13:18:40.106066 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:41.186427 (+1080361us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4800002958. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:18:41.904252 (+717825us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b54001fbdd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:41.904400 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16424,"prepare.queue_time_us":425795,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":6011,"raft.queue_time_us":22386,"raft.run_cpu_time_us":4084,"raft.run_wall_time_us":1133910,"spinlock_wait_cycles":214313088,"thread_start_us":312255,"threads_started":2,"wal-append.queue_time_us":86406,"wal-append.run_cpu_time_us":4965,"wal-append.run_wall_time_us":1329452}
I20260709 13:18:42.057282 17413 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:42.058318 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 454) took 1544 ms. Trace:
I20260709 13:18:42.058787 17413 rpcz_store.cc:276] 0709 13:18:40.536492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.550027 (+ 13535us) service_pool.cc:224] Handling call
0709 13:18:41.972739 (+1422712us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfeee 0000000000375939 00007f3284b331e0 00007f3284bfc0c4 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:42.026697 (+ 53958us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.057936 (+ 31239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.058243 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28815232}
I20260709 13:18:42.064069 19393 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.067947 19408 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:42.113359 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 361970us
I20260709 13:18:42.117823 19314 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:42.118605 19314 consensus_queue.cc:1243] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:18:42.157094 16741 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:42.157130 16743 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:42.158942 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:42.154249 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:42.166925 16744 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:42.147708 19083 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.145728 19393 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.154590 19314 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:18:42.188485 19314 consensus_queue.cc:1243] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:42.160969 19345 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:42.162240 19400 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:42.164829 19143 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.178009 19405 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.200927 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.203279 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.204100 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:42.206290 16740 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:42.211210 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:42.215222 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.211771 16737 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:42.226667 16740 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:18:42.228926 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.210440 19351 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:42.230815 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.199071 19345 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.196594 19441 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:18:42.237675 16740 raft_consensus.cc:1217] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-6.4]   Dedup: 4.3->[6.4-6.4]
I20260709 13:18:42.203922 19143 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.240279 16740 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:18:42.224920 19408 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.201108 19400 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.259724 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:42.252983 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 506) took 1260 ms. Trace:
I20260709 13:18:42.255986 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 503) took 1264 ms. Trace:
I20260709 13:18:42.257305 17414 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:42.261224 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.238409 18908 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.264267 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:42.274475 16740 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.272523 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.282531 17420 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:42.287753 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.288594 16737 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:42.290428 16745 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.291173 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.291744 16741 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:42.292480 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:42.292995 16743 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.293457 19400 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:42.294131 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.294921 16737 raft_consensus.cc:2410] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:18:42.295370 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.293362 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.295918 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.296290 19143 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.263772 19308 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:42.296672 16742 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:42.297331 19308 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:42.298204 19308 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:42.301956 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 468321us
I20260709 13:18:42.302068 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.302914 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.303498 16741 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:42.306152 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:42.270393 19437 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:18:42.311841 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.317838 17420 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:42.320250 17374 leader_election.cc:400] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:42.322033 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2408) took 1288 ms. Trace:
I20260709 13:18:42.323292 17374 leader_election.cc:403] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:18:42.273082 19431 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:42.323336 16595 rpcz_store.cc:276] 0709 13:18:41.033349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.472782 (+439433us) service_pool.cc:224] Handling call
0709 13:18:41.475605 (+  2823us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:41.780245 (+304640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.780295 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:41.787383 (+  7088us) log.cc:844] Serialized 52 byte log entry
0709 13:18:41.837247 (+ 49864us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:41.919208 (+ 81961us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.321799 (+402591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.321859 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.321885 (+    26us) raft_consensus.cc:1685] finished
0709 13:18:42.321896 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.321992 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":702757,"prepare.queue_time_us":108599,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":2077,"raft.queue_time_us":707683,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":6032,"thread_start_us":40727,"threads_started":1,"wal-append.queue_time_us":63695}
I20260709 13:18:42.310655 19393 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:42.319134 19362 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:42.330376 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:42.325366 19304 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:42.325802 19431 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:42.331720 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.331959 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 207) took 1369 ms. Trace:
I20260709 13:18:42.284329 18908 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.335597 16669 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:18:42.348312 16741 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:18:42.337347 16591 rpcz_store.cc:276] 0709 13:18:40.962289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.421530 (+459241us) service_pool.cc:224] Handling call
0709 13:18:41.433202 (+ 11672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.747941 (+314739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.747990 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:41.748388 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:18:41.797354 (+ 48966us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:41.808588 (+ 11234us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.331642 (+523054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.331725 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.331763 (+    38us) raft_consensus.cc:1685] finished
0709 13:18:42.331776 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.331902 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":843244,"prepare.queue_time_us":11174,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":2626,"raft.queue_time_us":839116,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":48249,"threads_started":1,"wal-append.queue_time_us":89683}
W20260709 13:18:42.338790 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.323649 19447 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:18:42.344847 17420 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:42.346349 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.353629 16742 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:18:42.346776 16663 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:42.346980 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:42.349097 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:42.355091 16745 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.349550 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:42.356261 16741 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.350274 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:42.359236 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.359537 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.361166 16742 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:42.361644 16737 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:42.360440 16744 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:42.262138 17411 rpcz_store.cc:276] 0709 13:18:40.991856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.993839 (+  1983us) service_pool.cc:224] Handling call
0709 13:18:41.471632 (+477793us) spinlock_profiling.cc:243] Waited 6.02 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:41.471846 (+   214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.532711 (+ 60865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.255767 (+723056us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54005802d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:42.255929 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138606720}
I20260709 13:18:42.362763 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.262372 17410 rpcz_store.cc:276] 0709 13:18:40.992908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.562759 (+569851us) service_pool.cc:224] Handling call
0709 13:18:41.663409 (+100650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.694197 (+ 30788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.252810 (+558613us) spinlock_profiling.cc:243] Waited 81.2 ms on lock 0x7b54000e33d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:42.252929 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186648832}
I20260709 13:18:42.339300 19431 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.347486 19387 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:42.353500 19362 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:42.333405 19289 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:18:42.352630 19409 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:18:42.336380 19440 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:18:42.395797 16741 raft_consensus.cc:2410] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:18:42.333788 19441 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:18:42.352908 19288 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:18:42.367501 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:42.364380 19308 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:42.391417 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.392557 19345 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:42.368535 18908 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.355749 19351 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:18:42.402253 16745 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:42.368850 19268 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.370963 19298 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:42.404891 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.396441 19212 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.405994 16744 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:42.410804 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.367736 19406 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:18:42.414419 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.399881 19382 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.417296 19212 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.416441 19268 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.418254 19298 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.432261 16666 raft_consensus.cc:2410] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:42.421728 19345 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:42.404910 19310 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:42.408725 17414 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.388952 19387 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.410959 19143 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:18:42.371111 19316 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:18:42.440346 16743 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:42.433012 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:42.450742 19388 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.456445 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:42.457206 16744 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:42.438093 19345 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.463225 19254 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:18:42.466741 19400 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:42.467324 19408 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:42.470222 17410 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:42.470579 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 204) took 2397 ms. Trace:
I20260709 13:18:42.476151 19394 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:42.476545 19394 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:42.476660 19387 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.477460 19394 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.479414 19394 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:42.479894 19297 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:42.487308 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 125385us
I20260709 13:18:42.427306 19382 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.488459 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.439570 19310 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.470969 16590 rpcz_store.cc:276] 0709 13:18:40.072899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.208342 (+1135443us) service_pool.cc:224] Handling call
0709 13:18:41.208731 (+   389us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:41.212080 (+  3349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:41.212122 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:41.243927 (+ 31805us) log.cc:844] Serialized 52 byte log entry
0709 13:18:41.378420 (+134493us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:41.387911 (+  9491us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.392103 (+  4192us) log.cc:844] Serialized 14 byte log entry
0709 13:18:41.747424 (+355321us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b1800221500. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:18:41.747616 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.747688 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.386527 (+638839us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b1800221500. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:42.470389 (+ 83862us) raft_consensus.cc:1685] finished
0709 13:18:42.470409 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.470529 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":352796,"prepare.queue_time_us":26164,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":360686,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":30811,"spinlock_wait_cycles":137018368,"thread_start_us":118121,"threads_started":2,"wal-append.queue_time_us":132831}
I20260709 13:18:42.496701 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.471172 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:42.498822 19449 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:18:42.488024 19385 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:42.497756 19441 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:42.439041 19362 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.518679 19408 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:42.518603 19351 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:42.508742 18908 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:42.512473 19254 consensus_queue.cc:1243] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:18:42.521672 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 504) took 1523 ms. Trace:
I20260709 13:18:42.547405 18908 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.517949 19385 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.497910 19288 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:18:42.499680 19409 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:18:42.526364 19382 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:42.527771 19345 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.529034 17419 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:42.530545 16634 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:42.531330 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.532135 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:42.559854 16590 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:42.560810 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.561286 16590 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.536804 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.517576 19441 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.539790 19268 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.541039 19298 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.543745 16591 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:18:42.545326 19438 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.715s	user 0.000s	sys 0.004s
I20260709 13:18:42.546157 17414 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:42.575157 19395 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:42.546933 17410 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:42.541880 19408 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:42.547477 19314 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:42.579860 19314 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:42.580964 19314 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.582490 19314 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.523386 19310 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.549043 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.584674 19395 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:42.584800 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.585120 19395 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.585556 19395 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.549059 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.547854 17416 rpcz_store.cc:276] 0709 13:18:40.992208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.992430 (+   222us) service_pool.cc:224] Handling call
0709 13:18:42.512929 (+1520499us) spinlock_profiling.cc:243] Waited 7.48 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:18:42.514271 (+  1342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.514459 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:42.514480 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:42.514517 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:42.514552 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.514607 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.516008 (+  1401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17192832}
I20260709 13:18:42.552641 16593 pending_rounds.cc:85] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 1
I20260709 13:18:42.586349 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.587343 16593 pending_rounds.cc:107] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 5.2
I20260709 13:18:42.587708 16593 raft_consensus.cc:2889] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.587843 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.556128 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2404) took 2230 ms. Trace:
I20260709 13:18:42.549620 18908 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.547832 19254 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:42.588461 16588 rpcz_store.cc:276] 0709 13:18:40.326000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.248016 (+922016us) service_pool.cc:224] Handling call
0709 13:18:41.263728 (+ 15712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:41.469596 (+205868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.555853 (+1086257us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b54000d9dd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:42.556049 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472063744}
I20260709 13:18:42.549793 19387 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.589550 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.588071 16593 pending_rounds.cc:107] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 7.3
I20260709 13:18:42.599395 16593 raft_consensus.cc:2889] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:42.602165 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:42.570808 19441 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.590390 19268 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:42.611008 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:42.625543 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.627475 19284 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:42.634322 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:42.614969 19373 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.145s	user 0.009s	sys 0.017s
I20260709 13:18:42.653661 17416 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:42.597874 19254 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:42.614276 19374 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:42.646031 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.613531 19289 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:42.613325 19414 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:42.647697 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.608244 19310 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:42.656441 16744 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:42.646852 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:42.608247 19452 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:18:42.659735 16737 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:42.657948 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.540959 19431 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.661326 16588 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:42.658593 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:42.647522 19176 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:42.645524 17410 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:42.612843 19387 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.646267 19298 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:42.664166 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:42.653463 19373 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.010s	sys 0.017s
I20260709 13:18:42.665163 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.629441 19307 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:18:42.655189 19358 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:18:42.629312 19363 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:18:42.658886 19374 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:42.657934 19414 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:42.660406 19452 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.669061 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.654932 19388 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:18:42.671377 16591 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:42.677381 16738 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:42.692865 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.668377 19406 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:18:42.695667 19429 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.900s	user 0.006s	sys 0.000s
I20260709 13:18:42.695591 16590 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:42.683542 19289 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:42.697086 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2385) took 3021 ms. Trace:
I20260709 13:18:42.697898 19289 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.696820 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.701313 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.702157 16593 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:42.703163 19450 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.303s	user 0.002s	sys 0.000s
I20260709 13:18:42.703960 19381 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.704427 19381 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.706558 19381 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.697504 16592 rpcz_store.cc:276] 0709 13:18:39.675439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.945303 (+269864us) service_pool.cc:224] Handling call
0709 13:18:39.945724 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:40.023965 (+ 78241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:40.023997 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:40.024598 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 13:18:41.551398 (+1526800us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b18003968e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:18:41.728893 (+177495us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:41.728930 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:41.728992 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.695101 (+966109us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:42.696868 (+  1767us) raft_consensus.cc:1685] finished
0709 13:18:42.696887 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:42.697028 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39766,"raft.queue_time_us":80087,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":10541,"spinlock_wait_cycles":12101376,"thread_start_us":177311,"threads_started":1,"wal-append.queue_time_us":1689528}
I20260709 13:18:42.702663 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:42.708148 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.709160 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.719981 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.681454 19374 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.722438 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.723901 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.723964 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.724679 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:42.725318 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:42.726665 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:42.676445 19379 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:42.681310 19414 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:42.676558 19383 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:42.730809 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.683604 19452 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:42.783692 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63626.2us
W20260709 13:18:42.784368 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 81789.8us
I20260709 13:18:42.693681 19212 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.788661 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.785185 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:42.792476 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.793540 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.789369 19212 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:42.798152 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:42.799203 16592 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:42.800825 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.802330 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.720204 19406 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:42.785739 19347 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.457s	user 0.034s	sys 0.089s
I20260709 13:18:42.802439 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:42.813290 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:42.827786 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:42.829468 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.829816 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.831259 19298 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:42.781163 19379 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:42.794637 19362 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:42.787982 19284 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:42.788661 19385 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.833755 19178 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:42.828434 19405 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:42.788130 19046 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:18:42.711387 19254 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:42.833926 19347 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.503s	user 0.034s	sys 0.090s
I20260709 13:18:42.839393 19379 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:42.839190 19288 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:42.848063 19178 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.843187 19358 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:42.844906 19388 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:42.877030 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 80452.4us
I20260709 13:18:42.862753 19358 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:42.862360 19288 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:42.844566 19284 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:42.878307 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 108281us
I20260709 13:18:42.879002 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.840539 19406 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.871752 19254 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.869825 19454 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:42.872871 19212 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:42.880483 19358 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.881769 19288 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.878047 19431 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.876192 19405 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:42.888962 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:42.890928 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 88550.1us
I20260709 13:18:42.885040 19385 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:42.894523 19284 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.894224 19212 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.898368 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.901074 19387 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.906226 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:42.873049 19388 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:42.912598 19178 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.914448 19448 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:42.889936 19454 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.987394 19454 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:42.987818 19288 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.991551 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.992625 16662 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:42.992841 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.993690 16591 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:42.929584 19409 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:42.995304 19409 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:42.996286 19409 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:42.997541 19409 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:42.932054 19460 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:18:42.935876 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48735.8us
I20260709 13:18:42.939183 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.000932 19452 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.001283 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.002091 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 210) took 2038 ms. Trace:
I20260709 13:18:43.002466 16589 rpcz_store.cc:276] 0709 13:18:40.963454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.473020 (+509566us) service_pool.cc:224] Handling call
0709 13:18:41.502758 (+ 29738us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:41.780079 (+277321us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:18:41.794963 (+ 14884us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.301590 (+506627us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b18002f2f60. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f3283ab858f 00007f327caf7571 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd
0709 13:18:42.538154 (+236564us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b18002f2f60. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f78423a 00007f327fbeea2a
0709 13:18:42.692481 (+154327us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:42.702850 (+ 10369us) log.cc:844] Serialized 26 byte log entry
0709 13:18:42.720667 (+ 17817us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:42.720904 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 13:18:42.721085 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:42.721162 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:42.931489 (+210327us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:43.001888 (+ 70399us) raft_consensus.cc:1685] finished
0709 13:18:43.001907 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.002030 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":292047,"prepare.queue_time_us":9421,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":3924,"raft.queue_time_us":296293,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":77664,"spinlock_wait_cycles":21765760,"thread_start_us":133452,"threads_started":1,"wal-append.queue_time_us":1019439}
I20260709 13:18:43.004650 19147 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.004839 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.005997 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.006465 16740 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:42.978621 19448 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:43.007773 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.007473 19448 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.008394 16742 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:43.008739 19448 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:42.978075 19311 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.007982 19388 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.952986 19471 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:18:42.985080 19395 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.009406 16595 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:43.010082 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.991456 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:43.019356 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:42.989951 19288 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.021934 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.022572 16591 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:43.023563 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:42.986548 19379 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.012883 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.013626 18996 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.013782 19452 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.007484 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.032694 16740 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:42.952044 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:42.903316 18908 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.030741 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:43.033896 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 81943.4us
I20260709 13:18:43.038401 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:42.952746 19385 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.040376 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.047817 16591 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:43.043049 18996 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.051129 19385 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.052003 16743 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 7 index: 9. (term mismatch)
I20260709 13:18:43.052385 16743 pending_rounds.cc:85] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 8
I20260709 13:18:43.052615 16743 pending_rounds.cc:107] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 6.9
I20260709 13:18:43.052888 16743 raft_consensus.cc:2889] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.056504 18908 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.014056 19311 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:43.058248 19395 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:42.956830 19374 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.059103 16589 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:43.059288 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.059932 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:43.060545 19311 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.060811 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.061100 19374 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:43.061877 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.062424 16592 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:42.978173 19358 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.006501 19441 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.073366 16740 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:18:43.073643 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:43.065881 19452 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:42.936060 19405 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.044256 19472 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:43.070842 19289 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.087891 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.072908 19374 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.092031 19374 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.094223 19374 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.103941 19379 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.075969 19452 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.104992 19452 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.108166 18996 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.111454 19405 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.112295 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.113379 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.113196 19414 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.114400 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.115252 16590 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:43.077853 19292 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:18:43.116956 19358 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.117553 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.118141 16590 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:43.121212 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.121862 16663 raft_consensus.cc:2368] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
I20260709 13:18:43.125222 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.125828 16669 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:43.126636 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.091259 19471 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.127202 19471 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.127525 19471 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.127816 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.128587 16667 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:43.135991 19260 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:43.136428 19260 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:43.137164 19441 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.137272 19260 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.138378 19260 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:43.138742 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50903.4us
I20260709 13:18:43.139406 19268 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.139899 19268 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.142055 19268 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.090309 19472 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:43.143126 19472 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.143898 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.144645 16668 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:43.146623 19212 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.147033 19212 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.147384 19212 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:43.149494 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.157198 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.158035 19454 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.158463 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.158586 19454 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:43.160758 19462 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 13:18:43.161195 19456 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:43.161991 16666 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1140) took 3135 ms (client timeout 1698 ms). Trace:
I20260709 13:18:43.179641 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.180205 16662 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:43.163555 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.164108 18996 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:43.165663 19253 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:43.166791 19046 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.193225 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.193616 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:43.167287 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.168119 19414 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.172480 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.197793 16742 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:43.197902 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.173139 19147 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.173697 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.176844 16596 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:43.177786 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.177945 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.180580 16662 pending_rounds.cc:85] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 7
W20260709 13:18:43.181586 16559 leader_election.cc:336] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.698s (SENT)
I20260709 13:18:43.205389 16662 pending_rounds.cc:107] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:43.205678 16662 raft_consensus.cc:2889] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.206209 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.206753 16595 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.213909 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.217351 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.217551 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.183722 16664 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: [5.8-6.9]
W20260709 13:18:43.179315 16666 rpcz_store.cc:269] 0709 13:18:40.026611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:40.055572 (+ 28961us) service_pool.cc:224] Handling call
0709 13:18:43.161747 (+3106175us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b540000ffd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:43.161896 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5398614272}
I20260709 13:18:43.122650 19409 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:43.235334 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.235641 19409 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.236783 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.238379 19409 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.191071 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.260172 16741 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.195241 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:43.172829 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.199568 16670 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:18:43.261255 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66071.9us
I20260709 13:18:43.200477 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.201041 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.218799 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:43.219965 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.263309 17416 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:43.205476 19147 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.134677 19471 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.221452 19362 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:43.265156 16595 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:43.266047 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.189792 19253 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:43.266825 16670 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:43.266667 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.221850 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.267493 16595 raft_consensus.cc:2368] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:43.267409 19253 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.266726 16634 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:43.268369 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.223781 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.268953 16742 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:18:43.226042 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:43.227083 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:43.223049 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.265928 19362 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.233770 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.272637 16745 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.240182 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.244841 19358 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:43.245954 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.201993 19448 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:43.192636 19311 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.192585 19456 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:43.204334 19046 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.194720 19400 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.254169 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.256371 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 250) took 1724 ms. Trace:
I20260709 13:18:43.191074 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.258657 16594 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:43.262622 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.268083 16665 raft_consensus.cc:2368] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7.
I20260709 13:18:43.272949 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.231331 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.274765 16740 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.268728 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.285946 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.317719 19362 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.268988 19253 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.277365 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.280301 16742 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:18:43.285117 16745 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:43.321681 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.321827 16595 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.322772 16594 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
W20260709 13:18:43.319866 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88630.4us
I20260709 13:18:43.275374 18996 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:43.319361 16662 rpcz_store.cc:276] 0709 13:18:41.483244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:41.483350 (+   106us) service_pool.cc:224] Handling call
0709 13:18:41.483641 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.992096 (+1508455us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5800370dd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:18:43.207249 (+215153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.207691 (+   442us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839079168}
I20260709 13:18:43.275995 16741 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:43.272203 19470 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.318514 19405 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.326761 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.327491 19452 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.328101 19362 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:43.343382 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.328230 19314 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:43.328620 19480 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:43.330822 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.318459 19409 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:43.331470 16744 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:43.323925 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.333184 16740 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:43.353856 16592 raft_consensus.cc:2368] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:43.353710 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.355531 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.357354 17374 leader_election.cc:400] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:43.325418 19289 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.346159 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.347643 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:43.348438 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85869.6us
W20260709 13:18:43.349771 19483 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:18:43.344555 19254 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.318821 19147 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.319222 19472 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.360178 16740 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:18:43.346693 19466 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.466s	user 0.003s	sys 0.000s
I20260709 13:18:43.360180 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.328626 19454 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:43.361232 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.353731 19374 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:43.362843 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.346918 19470 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.368587 19046 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.375334 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.282737 19448 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.345822 16595 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:43.344138 19362 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:43.285200 19311 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:43.388273 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.354367 19431 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:43.390244 16671 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:43.392549 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:43.409199 17374 leader_election.cc:400] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5.
I20260709 13:18:43.396126 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.348991 19212 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.401921 16670 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:43.374850 19409 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:43.401562 19362 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.412528 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.394954 19382 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:43.407513 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.354696 19314 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.345626 19405 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.418959 19456 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:43.401916 19448 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:43.419893 19314 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.420346 19405 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.420940 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.404533 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:43.413358 16744 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:43.423823 16744 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:18:43.424787 19452 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.425019 16739 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:43.417649 16595 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:43.412256 19468 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.427755 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.430079 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.417896 16667 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.431344 16739 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:18:43.431852 16739 pending_rounds.cc:85] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 6
I20260709 13:18:43.398783 19374 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.391155 19472 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.432142 16739 pending_rounds.cc:107] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:18:43.406579 19471 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.433308 16739 raft_consensus.cc:2889] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.434010 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.436400 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.410408 19454 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.410642 19431 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:43.436579 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.438292 16670 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:43.442473 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.443704 16667 raft_consensus.cc:2410] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:43.433087 19468 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.457847 16595 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
W20260709 13:18:43.458773 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.458925 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.460114 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.437966 19456 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.437539 19254 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:43.440922 19431 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.468876 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.472621 16667 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:43.437013 19452 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:18:43.475179 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.460668 19314 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.482091 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:18:43.483261 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 69084.8us
I20260709 13:18:43.487421 16587 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.464985 19468 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:43.493036 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 83175us
I20260709 13:18:43.498517 16667 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:18:43.499229 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:43.499456 17412 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:43.501560 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:43.502203 17412 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:43.490746 19455 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.490825 19373 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:43.505025 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:43.499780 19254 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:43.508108 17413 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.507911 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.470853 19409 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.511462 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.518963 16587 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:43.525704 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.493513 19314 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.512094 19454 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.534536 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.535758 16666 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.521965 19400 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.536182 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.538901 16667 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:43.544842 19212 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.548548 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:43.551249 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.552054 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.524680 17410 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:43.538443 19400 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.556237 19400 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.556520 19298 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.558321 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.559119 19374 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.559222 19298 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:43.560161 19472 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.560611 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.554075 19311 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.561580 19311 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.562317 19408 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.563175 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.563416 19408 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.560693 19310 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.572860 19362 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.577764 19400 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.574478 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.586629 16588 raft_consensus.cc:2376] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:18:43.586246 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.573881 19314 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.572649 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.540503 19470 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:43.603554 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 85993.3us
W20260709 13:18:43.604247 16708 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.736s (SENT)
I20260709 13:18:43.609098 16666 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:43.587989 16667 raft_consensus.cc:2368] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:18:43.590041 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.585777 16670 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:43.578215 19448 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.593567 19400 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:43.600301 19207 raft_consensus.cc:993] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
W20260709 13:18:43.591590 19492 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:18:43.614392 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.584118 19310 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.623067 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.629395 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.630131 16666 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:43.623713 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.630251 16588 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.632025 16669 raft_consensus.cc:2410] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:43.637151 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:43.639670 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 89418.3us
I20260709 13:18:43.641052 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.640458 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.642002 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.642674 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.642016 16666 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:43.643860 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.642728 17415 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 4: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:43.644506 16595 raft_consensus.cc:2368] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:43.645711 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:43.647203 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76850.2us
I20260709 13:18:43.648602 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.649345 16594 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:43.652722 16594 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:43.632658 19310 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.660094 19469 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:43.660567 19469 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:43.661332 19469 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.662477 19469 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:43.665809 17374 leader_election.cc:400] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:43.666280 17374 leader_election.cc:403] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:43.667623 19046 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.672523 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2374) took 1515 ms. Trace:
I20260709 13:18:43.655376 17415 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.654839 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.672416 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.674698 16593 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.675071 16667 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:43.672920 16738 rpcz_store.cc:276] 0709 13:18:42.157228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.157342 (+   114us) service_pool.cc:224] Handling call
0709 13:18:42.157582 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:42.692072 (+534490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.672274 (+980202us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54001c7354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:43.672464 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1194843904}
I20260709 13:18:43.680078 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:43.680152 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.681072 17420 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:43.682493 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.683296 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.673167 19455 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:18:43.690616 16588 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:18:43.695032 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.703436 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.703855 16593 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:43.695849 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1868) took 1015 ms. Trace:
I20260709 13:18:43.705319 19448 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:43.706213 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 334) took 1003 ms. Trace:
I20260709 13:18:43.696174 16592 raft_consensus.cc:1217] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[3.3-7.5]   Dedup: 3.3->[5.4-7.5]
I20260709 13:18:43.707523 16592 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 3
I20260709 13:18:43.707600 19288 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:43.707952 16592 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:18:43.708329 16592 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.681565 18996 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:18:43.695012 16671 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:43.710246 16667 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:43.705082 16589 rpcz_store.cc:276] 0709 13:18:42.680703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.006728 (+326025us) service_pool.cc:224] Handling call
0709 13:18:43.695699 (+688971us) spinlock_profiling.cc:243] Waited 30 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:43.695798 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69066112}
W20260709 13:18:43.707266 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 155265us
I20260709 13:18:43.711989 17411 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:43.713238 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.704882 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1883) took 1004 ms. Trace:
W20260709 13:18:43.714576 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.714010 16593 rpcz_store.cc:276] 0709 13:18:42.699977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.728943 (+ 28966us) service_pool.cc:224] Handling call
0709 13:18:43.672274 (+943331us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b5800010e98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:43.704806 (+ 32532us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3761920}
I20260709 13:18:43.716547 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.717145 16593 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.725258 16670 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:43.707126 16743 rpcz_store.cc:276] 0709 13:18:42.702533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.702666 (+   133us) service_pool.cc:224] Handling call
0709 13:18:42.702926 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.053310 (+350384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.705946 (+652636us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54003ed5d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:43.706143 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":801822080}
I20260709 13:18:43.729594 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:43.730317 16588 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:43.715430 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.732059 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.718137 16663 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.733866 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.696485 17409 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.737782 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.814687 16745 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:43.738092 16588 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:18:43.801589 19314 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:43.816483 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1869) took 1126 ms. Trace:
I20260709 13:18:43.817018 16588 rpcz_store.cc:276] 0709 13:18:42.690028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.737556 (+1047528us) service_pool.cc:224] Handling call
0709 13:18:43.737854 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:43.816077 (+ 78223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.816403 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:43.694852 19254 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:18:43.739450 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59439us
I20260709 13:18:43.821168 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:43.822767 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:43.696085 19362 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.715972 19472 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:18:43.739033 16663 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.825868 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.827370 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.828711 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.835891 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1885) took 1127 ms. Trace:
I20260709 13:18:43.836478 16587 rpcz_store.cc:276] 0709 13:18:42.708217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.834622 (+1126405us) service_pool.cc:224] Handling call
0709 13:18:43.835047 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.835460 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.835500 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:43.835559 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:43.835604 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.835679 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.835814 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:43.838521 16587 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:43.780243 16738 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:18:43.781966 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.783660 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.733544 19207 raft_consensus.cc:1081] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:18:43.793465 16593 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:43.879236 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.879861 16593 raft_consensus.cc:2376] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7. Re-sending same reply.
I20260709 13:18:43.880712 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:43.880870 19207 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:43.881240 16593 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:43.787606 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:43.797797 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 84636.7us
I20260709 13:18:43.883929 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.884423 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:43.885077 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.886061 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.886938 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.887097 19444 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:43.888000 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:43.888171 16593 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:43.888782 16668 raft_consensus.cc:2410] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:43.888695 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.889659 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.889765 16663 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:43.724540 19448 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:18:43.891083 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.782996 19311 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:43.797391 19450 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:43.709570 18996 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:43.840101 19441 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:43.844316 19456 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.847584 19468 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.786561 19381 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:43.779289 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.856726 19379 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:43.919323 19379 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.922081 19379 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.922672 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 472) took 1301 ms. Trace:
I20260709 13:18:43.923054 19381 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:43.923704 19381 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.850843 19431 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.925448 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.859939 17411 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:43.926301 16742 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:43.926326 17419 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:43.927289 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:43.911083 18996 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:43.931255 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2428) took 1403 ms. Trace:
I20260709 13:18:43.931658 16596 rpcz_store.cc:276] 0709 13:18:42.528102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.536557 (+  8455us) service_pool.cc:224] Handling call
0709 13:18:43.931102 (+1394545us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b540010e354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:43.931196 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1413813120}
I20260709 13:18:43.932466 19431 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:43.933903 16596 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:43.935159 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1876) took 1237 ms. Trace:
I20260709 13:18:43.934975 19454 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:43.874308 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:43.891319 19448 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:43.891522 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:43.892210 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.936389 19448 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:43.806133 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.892859 19506 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:43.937288 19448 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.938103 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:43.938477 19448 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.894331 19501 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:43.894652 19504 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:18:43.939424 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 65237.6us
I20260709 13:18:43.915051 19311 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.800637 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.941774 19311 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:43.942222 16744 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:18:43.942828 16744 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:43.943202 16744 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.897188 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.899375 16737 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:43.901075 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:43.947902 16741 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:43.949460 16670 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:18:43.903087 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.950626 19414 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:43.890682 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:43.951043 19207 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:18:43.886756 16662 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:43.849583 16589 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:43.951609 19414 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:18:43.951907 19207 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:43.952046 19414 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:43.930219 19373 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:43.953343 19373 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:43.953035 19414 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.953701 19373 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:43.954257 16589 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:43.935544 16591 rpcz_store.cc:276] 0709 13:18:42.698029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.037986 (+339957us) service_pool.cc:224] Handling call
0709 13:18:43.934991 (+897005us) spinlock_profiling.cc:243] Waited 6.94 ms on lock 0x7b540010de54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:43.935110 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15954304}
I20260709 13:18:43.954489 16596 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:43.954947 19400 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:43.955348 19400 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:43.956139 19400 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.957235 19400 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.954528 19373 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.958071 19373 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:43.902662 16587 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:43.950856 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47863.3us
I20260709 13:18:43.959092 19504 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:43.959873 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.959782 19504 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:43.942329 19311 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:43.959009 16589 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:43.961609 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:43.909973 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:43.970949 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 69199.3us
I20260709 13:18:43.931486 18996 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:43.863821 19502 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:18:43.220086 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:43.923161 17414 rpcz_store.cc:276] 0709 13:18:42.621174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.621667 (+   493us) service_pool.cc:224] Handling call
0709 13:18:42.653491 (+ 31824us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:43.175188 (+521697us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18002e7f80. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327de94291 00007f327deaf0ca 00007f327fbdd4ad 00007f327fbe3e1b
0709 13:18:43.175985 (+   797us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b2c0023b0b8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327de94291 00007f327deaf0ca 00007f327fbdd4ad 00007f327fbe3e1b 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:18:43.187791 (+ 11806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.187832 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:43.203480 (+ 15648us) log.cc:844] Serialized 52 byte log entry
0709 13:18:43.263861 (+ 60381us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:43.406471 (+142610us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.406767 (+   296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:43.922324 (+515557us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5400588c54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:43.922382 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:43.922431 (+    49us) raft_consensus.cc:1685] finished
0709 13:18:43.922445 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:43.922599 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":213166,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":2504,"raft.queue_time_us":1662,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":17062,"spinlock_wait_cycles":58484736,"thread_start_us":166733,"threads_started":2,"wal-append.queue_time_us":24884}
I20260709 13:18:43.968319 16744 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:43.986546 16739 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.906981 16663 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 13:18:43.976188 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:43.979959 16491 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 3 to 4, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 4 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:43.980726 18996 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:43.995332 17413 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:43.996721 19468 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:43.981895 19395 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:43.982625 19504 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:43.982401 19284 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:43.984169 16710 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:43.988564 16663 pending_rounds.cc:85] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 7
I20260709 13:18:44.009414 16663 pending_rounds.cc:107] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:18:44.009699 16663 raft_consensus.cc:2889] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:43.975342 19409 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:44.013695 16744 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:44.015491 16664 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:43.972591 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:43.914515 19441 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:43.985945 19414 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:18:44.034555 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62068.3us
I20260709 13:18:44.036978 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2440) took 1455 ms. Trace:
I20260709 13:18:44.044143 16590 rpcz_store.cc:276] 0709 13:18:42.581580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.651151 (+1069571us) service_pool.cc:224] Handling call
0709 13:18:43.651444 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.651644 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:43.677555 (+ 25911us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.036630 (+359075us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.036683 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:44.036733 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.036804 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.036925 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38920,"prepare.queue_time_us":28000,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"thread_start_us":319771,"threads_started":1}
I20260709 13:18:44.045217 16739 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:44.043708 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:44.050700 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:44.055042 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55772us
I20260709 13:18:44.056856 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.053207 19254 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.060914 19254 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:44.061381 19254 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.061154 16491 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 3 to 4, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.043007 19456 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.063728 19470 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:44.070240 19465 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:44.072041 19379 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:44.072643 19379 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.073232 19465 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:44.075021 19504 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.075556 19504 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.075829 19469 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:44.076335 19469 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:44.077642 19504 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:44.077263 19469 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.079485 19472 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:44.083299 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:44.084201 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.084895 19253 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.085294 19253 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.085726 19253 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:43.954922 19207 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.089644 19260 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:18:44.090302 19470 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:44.090555 19260 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:44.091030 19260 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:44.091840 19260 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.092989 19260 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:44.094982 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.094913 19379 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:44.095681 19469 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.100997 19510 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:18:44.102059 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51412.5us
I20260709 13:18:44.003301 19395 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:44.103183 19395 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.104712 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.106129 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:44.004042 19444 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.107041 19510 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.107383 19444 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:44.108040 19444 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:44.109076 19444 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.109740 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.032317 19441 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:44.127240 16494 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 218) took 27040 ms (client timeout 15000 ms). Trace:
I20260709 13:18:44.036502 19311 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:44.054507 19454 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:44.054664 19409 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:44.136121 19510 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:18:44.142458 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 494534us
I20260709 13:18:44.139832 16491 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 4, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.146549 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.148823 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.149736 16670 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:44.115227 19405 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:18:44.133291 19455 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:44.159818 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.128873 19286 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:18:44.180960 19409 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.145462 19510 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:44.183974 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:44.151078 19504 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:44.177668 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1867) took 1497 ms. Trace:
I20260709 13:18:44.179435 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.170491 16491 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.183800 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:44.127792 16494 rpcz_store.cc:269] 0709 13:18:17.016724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:17.017212 (+   488us) service_pool.cc:224] Handling call
0709 13:18:44.056901 (+27039689us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:38.588384 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:38.589075 (+   691us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:38.589127 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:18:38.589146 (+    19us) tablet.cc:662] Decoding operations
0709 13:18:38.589817 (+   671us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:38.589900 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:38.589928 (+    28us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:18:38.590463 (+   535us) tablet.cc:701] Row locks acquired
0709 13:18:38.590511 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:18:38.590692 (+   181us) write_op.cc:270] Start()
0709 13:18:38.590846 (+   154us) write_op.cc:276] Timestamp: P: 1783603118590630 usec, L: 0
0709 13:18:38.603596 (+ 12750us) op_driver.cc:348] REPLICATION: starting
0709 13:18:38.667234 (+ 63638us) log.cc:844] Serialized 1673 byte log entry
0709 13:18:39.155526 (+488292us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbdfe28 00007f328361a581 00007f3283619d58 00007f328361e9f1 00007f328361e9a9 00007f328361e939 00007f328361e901 00007f328361db2d 000000000038f354 000000000038f199
0709 13:18:39.261467 (+105941us) op_driver.cc:464] REPLICATION: finished
0709 13:18:39.362966 (+101499us) write_op.cc:301] APPLY: starting
0709 13:18:39.363062 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:39.363217 (+   155us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:39.363264 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:39.367778 (+  4514us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:39.368068 (+   290us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:18:39.368108 (+    40us) write_op.cc:312] APPLY: finished
0709 13:18:39.368407 (+   299us) log.cc:844] Serialized 41 byte log entry
0709 13:18:39.368574 (+   167us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:39.368660 (+    86us) write_op.cc:454] Released schema lock
0709 13:18:39.368717 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:39.370103 (+  1386us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:39.370133 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":11884,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":101119,"apply.run_cpu_time_us":2488,"apply.run_wall_time_us":7170,"num_ops":4,"prepare.queue_time_us":182061,"prepare.run_cpu_time_us":38793,"prepare.run_wall_time_us":575836,"raft.queue_time_us":311266,"raft.run_cpu_time_us":36470,"raft.run_wall_time_us":141503,"replication_time_us":599590,"spinlock_wait_cycles":5256064,"thread_start_us":516338,"threads_started":4,"wal-append.queue_time_us":120861,"wal-append.run_cpu_time_us":37351,"wal-append.run_wall_time_us":1609873}]]}
I20260709 13:18:44.193948 16594 rpcz_store.cc:276] 0709 13:18:42.680492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.653538 (+973046us) service_pool.cc:224] Handling call
0709 13:18:43.655092 (+  1554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:43.655532 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.655560 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:43.674620 (+ 19060us) log.cc:844] Serialized 26 byte log entry
0709 13:18:44.004510 (+329890us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:18:44.046734 (+ 42224us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.056794 (+ 10060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.056873 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.177410 (+120537us) raft_consensus.cc:1685] finished
0709 13:18:44.177432 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.177602 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9788,"prepare.queue_time_us":32013,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":24211,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":19072,"thread_start_us":294268,"threads_started":1,"wal-append.queue_time_us":492095}
I20260709 13:18:44.167742 17376 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.194841 16669 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:44.203817 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2401) took 1587 ms. Trace:
I20260709 13:18:44.204276 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2451) took 1079 ms. Trace:
W20260709 13:18:44.183266 19046 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.002s
I20260709 13:18:44.204281 16736 rpcz_store.cc:276] 0709 13:18:42.616608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:42.617216 (+   608us) service_pool.cc:224] Handling call
0709 13:18:42.617651 (+   435us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:42.618122 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:42.618159 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:42.653217 (+ 35058us) log.cc:844] Serialized 52 byte log entry
0709 13:18:43.797163 (+1143946us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b180038f2c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:18:44.037598 (+240435us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:44.037656 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.037756 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.183713 (+145957us) raft_consensus.cc:1685] finished
0709 13:18:44.183742 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.203757 (+ 20015us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12214,"raft.queue_time_us":453,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":30933,"spinlock_wait_cycles":5345664,"thread_start_us":240182,"threads_started":1,"wal-append.queue_time_us":1517335}
W20260709 13:18:44.222471 16592 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.000s	sys 0.018s
W20260709 13:18:44.222810 16592 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.000s	sys 0.018s
I20260709 13:18:44.224850 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:44.226603 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 85668.5us
I20260709 13:18:44.227201 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1873) took 1533 ms. Trace:
I20260709 13:18:44.227641 16588 rpcz_store.cc:276] 0709 13:18:42.693703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.818026 (+1124323us) service_pool.cc:224] Handling call
0709 13:18:43.818460 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.818742 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:43.862288 (+ 43546us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.203418 (+341130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.203480 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:44.203524 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.203589 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.203716 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7590,"prepare.queue_time_us":804,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":18546,"thread_start_us":323925,"threads_started":1}
I20260709 13:18:44.195518 19382 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:44.193643 19510 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:44.240025 16491 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.240561 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:44.242357 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:44.223977 19310 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:18:44.205205 19431 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:44.218842 19212 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:44.253037 16491 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 4 to 7, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 7 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.224363 19046 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7.
I20260709 13:18:44.169706 19455 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.239473 19382 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.239104 19500 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.181036 19311 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.223542 19501 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:18:44.236524 19307 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:18:44.285092 19382 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:44.292317 19455 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:44.255494 19310 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.254550 19212 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:44.295408 16595 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:44.285127 16491 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.344100 16491 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.294806 19310 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:44.285501 19500 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.249927 19254 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:18:44.368641 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 368557us
I20260709 13:18:44.369576 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.383244 19510 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.384624 18996 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:44.385479 19510 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:44.385404 18996 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.389954 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:44.392321 18996 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:44.413297 16491 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 2 to 3, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.360253 19500 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.434654 16491 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 2 to 4, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:44.342289 19381 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.439181 16491 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 4, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:44.483500 19518 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 13:18:44.484517 19516 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 13:18:44.444339 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:44.515152 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 274470us
I20260709 13:18:44.520975 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:44.515789 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 288534us
I20260709 13:18:44.485647 17410 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:44.535076 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74679.5us
I20260709 13:18:44.547005 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:44.548782 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.551666 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.555485 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.427661 19212 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.571393 16742 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 7. Ops: []
W20260709 13:18:44.519241 19400 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.000s
I20260709 13:18:44.574486 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.575603 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.577719 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:44.577432 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 299748us
I20260709 13:18:44.518677 19454 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:44.579317 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:44.579600 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.581564 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.591838 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.593813 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.596007 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.601979 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.571096 19307 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.606559 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:44.626227 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.641872 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.642670 16669 raft_consensus.cc:2376] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7. Re-sending same reply.
I20260709 13:18:44.649122 16736 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:44.651122 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.595408 19310 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.667351 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.651135 17376 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:44.672055 16666 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:44.629146 19212 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:44.675913 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:44.680011 16666 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:44.680521 16666 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:44.684410 18996 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:44.686548 19381 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:44.689925 19395 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
W20260709 13:18:44.690320 19260 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.000s
I20260709 13:18:44.691241 19254 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:44.692413 16708 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.727s (SENT)
I20260709 13:18:44.692884 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.700260 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:44.701085 16669 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:44.701792 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.701999 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:44.684690 19514 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:18:44.682723 19505 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:44.706673 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.686439 18996 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.726722 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.726936 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:44.699853 19448 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.749s	user 0.008s	sys 0.003s
I20260709 13:18:44.645817 19307 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.734488 16671 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:44.690369 19395 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.681443 16666 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.695278 19254 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.700928 19400 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:44.739637 16666 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:18:44.753264 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:44.756295 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.764174 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.766607 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:44.772385 16739 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:44.720585 19455 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.724640 19510 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:44.729363 19456 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:44.741029 19469 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.645s	user 0.007s	sys 0.004s
I20260709 13:18:44.776952 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:44.776276 16669 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:44.798583 16666 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:44.741557 19505 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:44.818490 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:44.786170 19514 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:18:44.780050 19537 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:44.778573 19408 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:44.782851 19510 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:44.836856 19510 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:44.839664 19523 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:18:44.839928 19395 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:44.841007 19527 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
W20260709 13:18:44.841723 19525 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
I20260709 13:18:44.781288 19530 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:44.727208 19454 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:44.843317 19517 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:18:44.817152 19409 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.845345 19441 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.000s
I20260709 13:18:44.845207 19409 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.845758 19441 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.841158 19505 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.847316 19409 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:44.847512 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1038) took 1115 ms. Trace:
I20260709 13:18:44.820991 19400 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:44.832252 19307 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:44.849319 16667 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:44.775844 19254 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.847915 16743 rpcz_store.cc:276] 0709 13:18:43.732243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.732732 (+   489us) service_pool.cc:224] Handling call
0709 13:18:43.733149 (+   417us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:43.734409 (+  1260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:43.734723 (+   314us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:43.739669 (+  4946us) log.cc:844] Serialized 52 byte log entry
0709 13:18:44.219449 (+479780us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:18:44.219489 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.219557 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.847269 (+627712us) raft_consensus.cc:1685] finished
0709 13:18:44.847327 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.847457 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2700,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15921,"thread_start_us":310726,"threads_started":1,"wal-append.queue_time_us":662399}
I20260709 13:18:44.839955 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2434) took 2286 ms. Trace:
I20260709 13:18:44.851881 16592 rpcz_store.cc:276] 0709 13:18:42.553480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.390471 (+836991us) service_pool.cc:224] Handling call
0709 13:18:43.695948 (+305477us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:43.708957 (+ 13009us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:43.709235 (+   278us) log.cc:844] Serialized 14 byte log entry
0709 13:18:43.858428 (+149193us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x7b1800396100. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:44.223217 (+364789us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:44.227940 (+  4723us) log.cc:844] Serialized 52 byte log entry
0709 13:18:44.228069 (+   129us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:44.542444 (+314375us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.586493 (+ 44049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.586589 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:44.837839 (+251250us) raft_consensus.cc:1685] finished
0709 13:18:44.837859 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.839805 (+  1946us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b540010d450. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:44.839898 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29319,"prepare.queue_time_us":464026,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":152508,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":17592,"spinlock_wait_cycles":719675520,"thread_start_us":827648,"threads_started":2,"wal-append.queue_time_us":651818}
I20260709 13:18:44.855022 16670 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:18:44.826939 19253 consensus_meta.cc:287] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.739s	user 0.006s	sys 0.006s
I20260709 13:18:44.823892 19471 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:44.832320 19529 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:18:44.827843 19456 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:18:44.892853 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 301098us
I20260709 13:18:44.893728 16559 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:44.874676 19506 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:44.901602 19506 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.903908 19506 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:44.874976 19253 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:44.915223 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.006s	sys 0.008s
W20260709 13:18:44.915859 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.006s	sys 0.008s
W20260709 13:18:44.916699 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 547195us
I20260709 13:18:44.920141 16596 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:44.896534 19448 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:44.929893 16665 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:44.877301 19471 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.934350 16743 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:44.934758 18996 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:44.965282 19503 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:44.966044 19503 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.968057 19253 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:44.970088 19254 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:44.971752 19500 consensus_meta.cc:287] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 13:18:44.972322 19500 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:44.984906 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65118.5us
I20260709 13:18:44.986635 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:44.987347 16737 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:44.989454 16737 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:44.968065 19503 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:44.997709 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:44.999333 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 482) took 1108 ms. Trace:
I20260709 13:18:44.999818 17410 rpcz_store.cc:276] 0709 13:18:43.891158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.891287 (+   129us) service_pool.cc:224] Handling call
0709 13:18:44.999144 (+1107857us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400299354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:44.999270 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809746304}
I20260709 13:18:45.001030 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.002189 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2459) took 1330 ms. Trace:
I20260709 13:18:45.002660 19558 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:45.002624 16596 rpcz_store.cc:276] 0709 13:18:43.671456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.956291 (+284835us) service_pool.cc:224] Handling call
0709 13:18:43.956560 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.920568 (+964008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.001952 (+ 81384us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b540010de54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:45.002129 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215634176}
W20260709 13:18:45.004457 19382 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.713s	user 0.010s	sys 0.000s
I20260709 13:18:45.004880 19382 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:44.973805 19400 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:45.007210 19400 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.009431 19400 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:45.011942 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.011837 19307 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:45.002192 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:45.014463 19524 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.347s	user 0.001s	sys 0.002s
I20260709 13:18:45.014885 19534 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:45.016506 19468 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:45.016966 19468 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:45.017465 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.018024 19373 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.017802 19468 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.018666 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.018779 19373 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:45.019136 19468 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:45.021564 19143 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:45.016392 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:45.022329 17410 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:45.014473 19307 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:45.023468 19307 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.023826 19307 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:45.024160 19510 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.025221 19472 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.014626 19506 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:45.026633 19506 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:45.027486 19546 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:45.027552 19506 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.028129 19546 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:44.935142 19448 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.028932 19506 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:45.029569 19494 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:45.030226 19546 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:45.030092 19494 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.030726 19546 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.032074 19494 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:45.031953 19546 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.032436 19448 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:45.033811 19558 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.033941 19546 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:45.034040 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2438) took 2453 ms. Trace:
I20260709 13:18:45.034363 19558 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:45.034485 16595 rpcz_store.cc:276] 0709 13:18:42.581097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.649613 (+1068516us) service_pool.cc:224] Handling call
0709 13:18:43.653300 (+  3687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:44.295891 (+642591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:45.033815 (+737924us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54001faed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:45.033983 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476163840}
I20260709 13:18:45.035082 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.035859 19373 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:45.036311 19373 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.035955 19558 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.036696 19373 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.039316 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:45.041257 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.042388 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.043031 19448 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:45.043762 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 150126us
W20260709 13:18:45.044759 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 127006us
I20260709 13:18:45.044960 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.053220 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.055881 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.057192 16559 leader_election.cc:400] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:18:45.057582 16559 leader_election.cc:403] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:45.059751 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.059885 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.061336 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.065299 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.064944 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.066478 16664 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:45.067320 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:45.072266 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.070061 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:45.073585 16665 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:44.217130 16593 rpcz_store.cc:276] 0709 13:18:43.124800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:43.888863 (+764063us) service_pool.cc:224] Handling call
0709 13:18:43.889115 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:43.889304 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:43.889611 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.188299 (+298688us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.194783 (+  6484us) log.cc:844] Serialized 14 byte log entry
0709 13:18:44.203953 (+  9170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:44.203999 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:44.204035 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:44.204101 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:44.204213 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15084,"prepare.queue_time_us":6497,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":63,"raft.queue_time_us":13332,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":10022,"spinlock_wait_cycles":122926080,"thread_start_us":295078,"threads_started":1}
I20260709 13:18:45.077742 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:45.081045 16594 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:45.082552 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:45.084537 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:45.085456 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.061944 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.046988 19441 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:45.056618 19311 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.054574 19448 raft_consensus.cc:2764] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:18:45.100709 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.092311 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.102380 16670 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:45.091792 16668 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:18:45.110272 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:45.110965 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:45.111727 16738 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.123855 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:45.124756 16737 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:45.061869 19254 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:45.077086 19400 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:18:45.080067 18996 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:45.062788 19510 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:45.136068 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88136.2us
I20260709 13:18:45.326449 16738 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:45.138012 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 93456.1us
I20260709 13:18:45.328879 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:45.138540 19537 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:18:45.148204 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:45.149389 16740 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:45.160120 16736 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:45.162562 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:45.080495 19382 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:45.080698 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:45.178256 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.332284 16593 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
W20260709 13:18:45.190438 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.198060 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.094063 19454 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.336228 16745 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:18:45.336760 16745 pending_rounds.cc:85] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 3
I20260709 13:18:45.337018 16745 pending_rounds.cc:107] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:45.337283 16745 raft_consensus.cc:2889] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:45.058271 19373 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.309506 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:45.341292 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.341568 16737 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:45.341655 17416 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:45.093868 19408 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:18:45.320423 16491 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.009s	sys 0.062s
I20260709 13:18:45.142936 19535 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:45.342195 16664 raft_consensus.cc:2410] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:45.342473 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:45.344136 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:45.358373 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.360448 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.362677 16671 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:45.360831 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:45.363606 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:45.364369 16666 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:45.349655 16491 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.009s	sys 0.063s
W20260709 13:18:45.366267 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 352852us
I20260709 13:18:45.367735 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:45.354658 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:45.377478 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.385630 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:45.392740 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.393559 16594 raft_consensus.cc:2376] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7. Re-sending same reply.
I20260709 13:18:45.312394 19448 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.408385 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.408744 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.410185 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.410455 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.410804 16592 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.411099 16590 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:45.170748 19507 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.197s
I20260709 13:18:45.338917 19311 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:45.327821 19400 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:45.380347 19382 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:45.412595 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.433231 16589 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.413816 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.434587 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.435182 16590 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 3.
I20260709 13:18:45.436213 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.437111 16590 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:45.437983 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.414183 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:45.380890 19212 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.695s	user 0.007s	sys 0.003s
I20260709 13:18:45.131940 19543 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.296s
I20260709 13:18:45.409441 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.352681 19530 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.349547 19510 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.363080 17410 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.418452 19310 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:45.349320 19551 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.214s	user 0.003s	sys 0.003s
I20260709 13:18:45.421260 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.423354 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.361610 19298 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:45.389691 19541 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:18:45.474488 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.483855 16596 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:45.484683 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.423993 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:45.428377 19500 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:45.485914 16742 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:45.427726 16595 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 4: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:45.487353 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.482005 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:45.489090 16588 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:18:45.428396 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 73817.7us
I20260709 13:18:45.490391 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:45.491850 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:45.405226 16666 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:45.388897 19506 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.368026 19454 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:45.378356 19405 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:45.440127 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:45.393280 19408 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:45.395081 19358 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:45.460441 16592 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:45.419677 19505 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:18:45.405896 19374 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:45.428524 19207 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.431090 19414 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:18:45.361964 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:45.426237 19448 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.472463 16593 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:45.565052 16593 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:45.566138 16593 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.567382 16593 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:45.470902 19529 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:18:45.471657 19522 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:45.509018 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:45.547443 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 147695us
I20260709 13:18:45.548235 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:45.469687 17409 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:45.471817 19307 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.481142 19310 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.577567 16591 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:45.479501 16589 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:45.522238 19506 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:45.474227 19554 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:45.494982 19379 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:45.593454 16743 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:45.600680 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.471743 19510 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:45.610610 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.522492 19471 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:45.632743 19373 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:45.486586 19298 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.634187 19506 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.634927 19535 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:45.572329 19519 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:45.627123 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:45.637126 18996 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:45.637617 18996 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:45.644388 19566 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:45.644989 19311 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.625950 19395 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:45.647686 19563 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 13:18:45.647513 19395 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.627568 16743 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:18:45.521620 19400 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.659044 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.603012 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:45.651964 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:45.689745 16662 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:45.614678 19559 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:45.706619 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47669.3us
I20260709 13:18:45.471698 19530 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:45.728207 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:45.635407 19535 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.571012 19207 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.750887 16596 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:45.754130 16592 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:18:45.754638 16592 pending_rounds.cc:85] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 5
I20260709 13:18:45.754935 16592 pending_rounds.cc:107] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:18:45.755283 16592 raft_consensus.cc:2889] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:45.755668 16592 pending_rounds.cc:107] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:18:45.755952 16592 raft_consensus.cc:2889] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:45.713165 19562 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.247s	user 0.000s	sys 0.004s
I20260709 13:18:45.639423 19554 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.705439 19543 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.759979 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:45.764495 19268 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:45.762303 19546 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:18:45.762130 19465 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 13:18:45.770848 19550 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:18:45.720230 19289 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:45.773474 19289 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.773854 19289 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:45.775267 19212 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:45.775936 19543 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.775959 19212 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:45.777532 19212 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.780118 19379 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:45.780661 19379 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:45.782208 19444 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:18:45.803447 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44224.3us
I20260709 13:18:45.589354 19522 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.897557 19373 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.905478 19373 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:45.749331 19408 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:45.752146 19454 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:45.940125 19569 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.442s	user 0.005s	sys 0.000s
I20260709 13:18:46.035169 19575 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.035862 19575 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.040609 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 235617us
I20260709 13:18:46.038493 19559 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.096315 19559 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.034320 19143 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.128221 19549 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.128963 19549 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.131628 19549 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:45.632534 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:46.191854 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 596356us
W20260709 13:18:46.192219 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 554587us
I20260709 13:18:46.193150 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:46.195446 16665 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.120s	user 0.019s	sys 0.011s
I20260709 13:18:46.196113 16665 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
W20260709 13:18:46.211867 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 169274us
I20260709 13:18:46.217947 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1845) took 1129 ms. Trace:
W20260709 13:18:46.219316 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.223388 16744 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.005s	sys 0.034s
W20260709 13:18:46.224967 16744 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.005s	sys 0.034s
W20260709 13:18:46.224447 16594 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.010s	sys 0.030s
W20260709 13:18:46.223618 17376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.037s	sys 0.060s
W20260709 13:18:46.230367 17376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.037s	sys 0.060s
W20260709 13:18:46.230690 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 787311us
W20260709 13:18:46.226986 16594 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.010s	sys 0.030s
W20260709 13:18:46.226003 17375 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.022s	sys 0.018s
W20260709 13:18:46.233125 16593 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.665s	user 0.018s	sys 0.007s
I20260709 13:18:46.218413 16665 rpcz_store.cc:276] 0709 13:18:45.069199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.069861 (+   662us) service_pool.cc:224] Handling call
0709 13:18:46.197876 (+1128015us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:46.244558 16559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.017s	sys 0.029s
W20260709 13:18:46.230059 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.244910 16559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.017s	sys 0.029s
W20260709 13:18:46.223097 19494 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.014s	sys 0.025s
W20260709 13:18:46.245460 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 753661us
W20260709 13:18:46.245568 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.232491 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:45.572170 19358 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.226420 16736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.000s	sys 0.010s
I20260709 13:18:46.241601 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:46.226984 17414 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.010s	sys 0.002s
I20260709 13:18:45.577337 16737 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:18:46.270416 17414 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.010s	sys 0.002s
W20260709 13:18:46.250788 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57703.4us
I20260709 13:18:46.271896 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.273515 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:46.254390 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.246565 16593 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:18:46.244433 17375 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.023s	sys 0.019s
W20260709 13:18:46.229986 19506 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.016s	sys 0.021s
W20260709 13:18:46.255260 19494 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.014s	sys 0.026s
I20260709 13:18:46.268123 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.230918 19532 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:46.292676 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:46.229159 19455 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:45.570017 19374 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.262988 19510 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:46.296446 19395 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.277103 19350 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:46.270334 19268 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.298764 16587 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:46.382591 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 90031.2us
I20260709 13:18:46.383661 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.228669 19533 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:46.239768 19310 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:46.289405 16736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.000s	sys 0.010s
I20260709 13:18:46.311093 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:46.394137 16665 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.255355 19298 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.262564 19546 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.397522 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:46.251952 19314 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:46.243618 19549 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:46.371256 16561 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:46.370222 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:46.402310 16668 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:46.374993 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 99982.1us
W20260709 13:18:46.376104 16588 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.886s	user 0.018s	sys 0.024s
W20260709 13:18:46.301136 19506 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.016s	sys 0.021s
I20260709 13:18:46.278136 19544 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.290385 19471 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.299499 19456 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.403851 16588 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:46.379078 19448 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:46.405387 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1048) took 1057 ms. Trace:
W20260709 13:18:46.375398 19558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.331s	user 0.008s	sys 0.007s
I20260709 13:18:46.374130 19307 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.414495 16665 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:46.405835 16561 leader_election.cc:403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:18:46.398315 19382 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.010s	sys 0.011s
I20260709 13:18:46.403501 19310 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.407207 19298 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:46.382694 19207 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.373128 19532 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.425873 19558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.382s	user 0.009s	sys 0.007s
I20260709 13:18:46.402957 19284 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:46.405623 19472 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.098s	sys 0.155s
I20260709 13:18:46.398483 19268 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.408571 19510 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.432827 19307 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.407244 19546 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.380363 19455 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.424369 19506 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:46.383605 19501 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.002s	sys 0.014s
I20260709 13:18:46.412215 19395 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:46.431932 19580 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 13:18:46.442385 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 209992us
I20260709 13:18:46.419394 19470 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.444536 16668 raft_consensus.cc:2410] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:18:46.414808 19559 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.419798 19530 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.423671 19543 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.375555 18996 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.426898 19554 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.421700 19582 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:18:46.405221 17410 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.918s	user 0.014s	sys 0.014s
W20260709 13:18:46.431550 19382 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.010s	sys 0.011s
W20260709 13:18:46.424074 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55296.1us
I20260709 13:18:46.570264 17410 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:46.429399 19522 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.407210 19314 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.404991 19535 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:46.408819 19549 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.580166 19589 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:18:46.435488 19575 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.409886 19533 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.424088 19456 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.446674 19284 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.559607 19506 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.451596 19594 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:46.561033 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:46.441603 19212 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:18:46.434549 19298 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.436000 19310 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:46.547133 19468 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.528s	user 0.013s	sys 0.018s
W20260709 13:18:46.560622 19501 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.126s	user 0.002s	sys 0.014s
W20260709 13:18:46.570792 17410 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 526) took 1509 ms (client timeout 1762 ms). Trace:
W20260709 13:18:46.599360 17410 rpcz_store.cc:269] 0709 13:18:45.061262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.061528 (+   266us) service_pool.cc:224] Handling call
0709 13:18:46.570725 (+1509197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:46.566746 19574 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:18:46.616628 19501 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:46.617089 19501 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:46.617933 19501 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.581655 19379 consensus_meta.cc:287] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.801s	user 0.016s	sys 0.011s
I20260709 13:18:46.582078 19284 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:46.420804 16736 rpcz_store.cc:276] 0709 13:18:45.347801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.591544 (+243743us) service_pool.cc:224] Handling call
0709 13:18:45.591911 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.592168 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:45.655195 (+ 63027us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.404768 (+749573us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.404968 (+   200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:46.405027 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:46.405086 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.405173 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.405318 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10410,"prepare.queue_time_us":165522,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":104,"thread_start_us":738544,"threads_started":1,"wal-append.queue_time_us":767546}
I20260709 13:18:46.572072 19382 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:18:46.449231 19472 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.098s	sys 0.156s
I20260709 13:18:46.591656 19310 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.593240 19532 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.595321 19506 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.596580 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1899) took 1192 ms. Trace:
I20260709 13:18:46.648661 16594 rpcz_store.cc:276] 0709 13:18:45.404473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.404683 (+   210us) service_pool.cc:224] Handling call
0709 13:18:45.405011 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:45.405238 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:45.405269 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:45.406408 (+  1139us) log.cc:844] Serialized 26 byte log entry
0709 13:18:46.232145 (+825737us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:46.232185 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.232248 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:46.596330 (+364082us) raft_consensus.cc:1685] finished
0709 13:18:46.596352 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.596502 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":761512,"threads_started":1,"wal-append.queue_time_us":1006418}
I20260709 13:18:46.569710 19559 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:46.587752 19554 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:18:46.611649 19374 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.598572 19373 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.566447 19470 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.613530 19298 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:46.676486 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 424830us
I20260709 13:18:46.678813 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 487) took 1527 ms. Trace:
W20260709 13:18:46.694633 19400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.002s	sys 0.017s
I20260709 13:18:46.704152 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 349) took 2234 ms. Trace:
I20260709 13:18:46.707144 17414 rpcz_store.cc:276] 0709 13:18:45.151486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.151620 (+   134us) service_pool.cc:224] Handling call
0709 13:18:45.151907 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.152159 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:45.379424 (+227265us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.678377 (+1298953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:46.678463 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:46.678522 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.678603 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.678730 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18802,"prepare.queue_time_us":352844,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":317939,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":1518313,"threads_started":3,"wal-append.queue_time_us":1013043}
I20260709 13:18:46.713788 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:46.722780 17410 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:46.668399 19501 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:46.726452 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.725282 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 316151us
I20260709 13:18:46.647476 19522 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:46.729367 19522 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.730809 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.549s (SENT) [suppressed 26 similar messages]
I20260709 13:18:46.731599 19522 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.735037 19532 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:46.737717 19532 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:46.721899 16744 rpcz_store.cc:276] 0709 13:18:44.443374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:44.443888 (+   514us) service_pool.cc:224] Handling call
0709 13:18:44.464827 (+ 20939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:44.466133 (+  1306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:44.466175 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:45.139749 (+673574us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b2c00141498. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:18:45.140089 (+   340us) log.cc:844] Serialized 26 byte log entry
0709 13:18:45.356319 (+216230us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:45.939485 (+583166us) spinlock_profiling.cc:243] Waited 54.7 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:46.226587 (+287102us) log.cc:844] Serialized 14 byte log entry
0709 13:18:46.677561 (+450974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:46.677671 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:46.677736 (+    65us) raft_consensus.cc:1685] finished
0709 13:18:46.677755 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:46.677963 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26671,"prepare.queue_time_us":219432,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":154193,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":24580,"spinlock_wait_cycles":135146112,"thread_start_us":1469668,"threads_started":3,"wal-append.queue_time_us":301254}
W20260709 13:18:46.739151 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:46.721966 19400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.003s	sys 0.017s
I20260709 13:18:46.723367 19362 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.647809 19379 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.704615 19559 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.758999 16591 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:18:46.760087 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:46.760605 16591 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:46.761312 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:46.798251 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:46.745134 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:46.802742 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 534677us
W20260709 13:18:46.798573 19289 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.987s	user 0.017s	sys 0.000s
I20260709 13:18:46.805893 16595 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:46.806720 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:46.755857 19362 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.814715 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 406223us
I20260709 13:18:46.813565 16589 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:46.830806 16589 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:46.816181 16596 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:46.831774 16589 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.833040 16589 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:46.800717 19350 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:18:46.815745 16559 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:46.835932 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.812719 19556 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.812932 17409 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:46.843565 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.817181 19546 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.812597 19522 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:18:46.852301 16589 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:46.839689 19395 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.856339 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.857856 16559 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.838551 19455 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.841300 19554 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.841507 19613 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.798746 19382 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.813838 19298 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:18:46.863948 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:46.877679 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:18:46.854681 19585 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
I20260709 13:18:46.815025 19289 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:46.864303 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 133578us
I20260709 13:18:46.845522 19268 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:46.860317 19582 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:46.862712 19374 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:46.862524 19395 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:46.902163 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:46.920775 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:46.851069 19594 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:18:46.848451 19556 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:46.901000 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.864367 19472 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:46.902662 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:46.890029 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:46.902298 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:46.897101 19598 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:18:46.901939 17410 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:18:46.930102 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52511.6us
W20260709 13:18:46.889441 19537 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.500s	user 0.000s	sys 0.018s
I20260709 13:18:46.927240 17414 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 7 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:18:46.933760 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:46.898721 19362 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.901417 19286 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:46.962417 16559 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.973428 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:46.888262 19379 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.895069 19610 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:46.984220 19610 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.888515 19470 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.903451 19350 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:46.884312 19494 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:46.985497 19614 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:18:46.985862 19494 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.959671 17416 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:46.972255 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:46.988590 19494 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:46.988802 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:46.989135 19494 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:46.899591 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:46.989615 19468 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:46.989849 17413 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:46.990036 19468 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:46.990159 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:46.990507 19468 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.991549 19577 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:46.992132 19577 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:46.993045 19577 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:46.977106 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 100846us
I20260709 13:18:46.993286 17416 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:46.991353 19494 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:46.995935 19494 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.978345 16589 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:46.931042 19444 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:46.999135 19444 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.999583 19444 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:46.999717 19468 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.001958 19468 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:46.931341 19582 raft_consensus.cc:2764] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:18:47.003547 19289 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.005861 19558 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:47.007529 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 446542us
I20260709 13:18:47.008811 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:47.010082 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:46.996896 19350 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:46.981205 16596 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.011263 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.012338 17414 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
W20260709 13:18:47.013703 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51350.5us
I20260709 13:18:47.014775 16559 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.962136 19143 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:46.938668 19599 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:47.027071 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.017845 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:47.027652 16664 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:47.019345 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:46.895244 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.022997 16740 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:47.022964 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.047055 16736 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:46.970993 19617 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.025722 16744 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:18:47.006384 19558 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:46.879245 19613 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.028429 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.020689 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:46.974730 19537 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:18:47.029148 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:47.029271 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.033648 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.034822 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.035473 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.024127 19554 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.023999 19530 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.036202 19374 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:47.035693 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.036748 19373 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:47.016146 19350 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.037640 16593 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:47.024021 19472 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.039750 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.041395 19604 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:47.041980 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:46.978650 19601 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:46.940799 19553 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 13:18:47.095343 17415 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:47.096526 19444 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.097837 19604 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:47.043110 17414 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:47.047274 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.051777 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:47.099282 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.099650 16666 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:47.055384 17409 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:47.059523 19454 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.059904 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1046) took 1712 ms. Trace:
I20260709 13:18:47.060559 19597 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:47.044600 19143 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.114512 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1051) took 1732 ms. Trace:
I20260709 13:18:47.118307 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.074441 16595 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.076066 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.047281 19611 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:18:47.077864 17412 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
W20260709 13:18:47.078977 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49888.9us
I20260709 13:18:47.081208 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:47.120831 16559 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:47.122422 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.026914 19599 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:47.011476 19286 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.050226 19559 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.090107 17420 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:47.065812 19617 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.102628 16590 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:47.063230 19284 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.101716 16737 rpcz_store.cc:276] 0709 13:18:45.347226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.576695 (+229469us) service_pool.cc:224] Handling call
0709 13:18:45.577001 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.577204 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:46.376649 (+799445us) log.cc:844] Serialized 14 byte log entry
0709 13:18:47.059578 (+682929us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.059630 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:47.059669 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.059735 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.059849 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":329679,"raft.queue_time_us":334250,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":352504,"threads_started":1,"wal-append.queue_time_us":498405}
I20260709 13:18:47.108003 19395 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.108132 19610 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.098323 19604 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:47.111871 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.102931 19455 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:47.123549 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.124897 16671 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:18:47.122632 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:47.129420 17411 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:47.115043 16741 rpcz_store.cc:276] 0709 13:18:45.383226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.612452 (+229226us) service_pool.cc:224] Handling call
0709 13:18:45.612708 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:45.612906 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:46.241253 (+628347us) log.cc:844] Serialized 14 byte log entry
0709 13:18:47.035191 (+793938us) log.cc:844] Serialized 14 byte log entry
0709 13:18:47.113850 (+ 78659us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.114165 (+   315us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:47.114218 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.114300 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.114443 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505973,"prepare.queue_time_us":168232,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":382117,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":930024,"threads_started":3,"wal-append.queue_time_us":503500}
I20260709 13:18:47.192862 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:47.131568 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:47.132992 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.194126 16669 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:47.144012 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.195209 16743 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:18:47.091421 19530 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:47.129410 19472 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:47.162781 16595 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:47.204488 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.205313 16595 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:47.099027 19510 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:47.208250 16667 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9.
I20260709 13:18:47.080359 19594 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:47.166630 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97105.3us
W20260709 13:18:47.211705 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 89175us
I20260709 13:18:47.212348 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:47.119500 16739 raft_consensus.cc:2410] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:18:47.085350 19268 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:18:47.176434 17410 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:47.190838 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.150676 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.088095 19207 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:47.144385 19289 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:47.113926 19454 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.140074 19599 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:47.210657 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.217958 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:47.166826 19448 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.207767 19455 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.140710 19537 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:18:47.169884 18996 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.223469 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.267690 16594 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:18:47.190250 19639 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.280076 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.221190 19379 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:47.225170 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.281512 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:47.226913 16589 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:47.140558 19597 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.199261 19643 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.238715 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.239466 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:47.199605 19572 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.240417 16663 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.240839 16739 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:47.222116 19559 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:47.267891 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.215444 19510 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.224854 16708 leader_election.cc:400] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:47.226539 19289 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.286944 19455 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.314380 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.287813 19284 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.293453 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:47.317777 17420 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.297714 19611 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.298256 16669 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:47.298910 16671 raft_consensus.cc:2368] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
W20260709 13:18:47.300037 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:47.284974 19373 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:47.303336 16743 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:47.262181 19595 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.325541 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:47.262475 19594 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:47.304580 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.326395 16669 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:47.327131 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.280524 19582 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.209257 19530 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.329535 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:47.319060 16738 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:47.288196 19617 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.325748 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.314459 19289 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.328647 17420 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:47.328909 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.335263 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.336033 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.336581 16671 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:47.330101 16742 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.337357 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.279837 18996 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.269677 19448 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:47.316881 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 92162.3us
I20260709 13:18:47.339010 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.339704 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:47.319938 19455 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.321776 19444 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.341833 16669 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:47.330279 19559 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.346256 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.333616 19582 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:47.339638 19624 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:18:47.353977 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:47.357870 17420 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.360073 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.365597 16663 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:18:47.367086 16663 raft_consensus.cc:2955] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:47.355837 16669 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.330008 19611 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.338316 19558 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.319905 19641 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:18:47.289788 19639 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.377858 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:47.384394 16742 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:47.309376 19643 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:47.372618 19638 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:18:47.312366 19572 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.400022 16671 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:47.372576 19582 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.388257 19558 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:47.350432 19455 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.383539 19143 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.333791 19595 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.356839 19559 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.391371 19286 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.370880 19650 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:18:47.468556 18996 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.469151 18996 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.469614 18996 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.469630 19314 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.471527 18996 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.475731 19597 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.476258 19597 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:47.476824 19510 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.476728 19597 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.477877 19530 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.442655 19611 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.499497 19454 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.528102 16741 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:47.528748 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.529314 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.529848 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.529925 16741 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:47.530939 16742 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:18:47.531461 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.531885 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:47.532625 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.532835 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.533205 16741 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
I20260709 13:18:47.533962 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.534130 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.533344 16742 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:47.535198 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.535745 16742 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:47.536653 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.537277 16742 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:47.539979 16743 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:47.533115 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:47.543387 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:47.399034 19373 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:47.558837 19572 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.563105 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.563968 17416 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:47.565829 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.540078 19350 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.623772 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 291) took 1333 ms. Trace:
W20260709 13:18:47.569765 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 197249us
I20260709 13:18:47.572599 16742 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:47.655332 16662 rpcz_store.cc:276] 0709 13:18:46.290656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.291157 (+   501us) service_pool.cc:224] Handling call
0709 13:18:46.374734 (+ 83577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.582534 (+207800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:46.582572 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:46.582615 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:46.582654 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.623494 (+1040840us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54001f7054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:47.623558 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.623704 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477435264}
I20260709 13:18:47.656430 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:47.596820 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.651178 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.609113 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.596155 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:47.657693 16738 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:47.657965 16743 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:47.604360 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.624713 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 292) took 1333 ms. Trace:
I20260709 13:18:47.656690 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.659087 16670 rpcz_store.cc:276] 0709 13:18:46.291925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.292076 (+   151us) service_pool.cc:224] Handling call
0709 13:18:46.369790 (+ 77714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:46.581527 (+211737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:46.581558 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:46.581602 (+    44us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:46.581645 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.624457 (+1042812us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b540009ef54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:47.624516 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:47.624651 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486468864}
I20260709 13:18:47.659996 16742 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:47.598145 16663 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 13:18:47.627519 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:47.542559 19636 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 13:18:47.562604 19633 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 13:18:47.569511 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.700546 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:47.701315 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 73891.3us
I20260709 13:18:47.702307 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:47.709271 19643 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.716985 16743 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:47.539515 19604 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.706722 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:47.648519 19644 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:18:47.720410 16668 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:47.722191 16662 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:18:47.622984 19558 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:47.606588 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:47.733067 16745 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:47.728804 19639 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:47.739192 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 368409us
I20260709 13:18:47.569159 19653 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:47.586232 19645 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:47.743854 19604 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.744139 19645 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.586460 19597 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.719859 19603 raft_consensus.cc:2764] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:47.706563 19212 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:47.743666 19653 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.719086 16742 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:47.751283 19558 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.756412 19659 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:47.758092 19659 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:47.758517 19659 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:47.761716 19597 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:47.762300 19597 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.764212 19597 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.745800 19455 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:47.748670 19374 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.002s
I20260709 13:18:47.747421 19559 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:47.725479 19655 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:18:47.746703 19653 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:47.773394 19595 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.775342 19374 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:47.774437 19645 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.786934 19657 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:47.788388 19657 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:47.790640 19657 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:47.791178 19599 consensus_meta.cc:287] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.002s
I20260709 13:18:47.791618 19599 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.793275 19599 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:47.794124 19653 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.796490 19653 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:47.796751 19659 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:47.856103 19659 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:47.896474 19648 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.590s	user 0.004s	sys 0.000s
W20260709 13:18:48.830379 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800225ac0 after lost signal to thread 15479
W20260709 13:18:48.917852 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 1.21717e+06us
I20260709 13:18:48.948463 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1054) took 1420 ms. Trace:
I20260709 13:18:48.948990 16745 rpcz_store.cc:276] 0709 13:18:47.527968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.533745 (+  5777us) service_pool.cc:224] Handling call
0709 13:18:48.947885 (+1414140us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540005ad54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:48.948175 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456494976}
I20260709 13:18:48.980269 19408 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:48.981492 19408 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:48.983671 19408 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:48.984227 19408 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:48.984725 19408 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:48.986390 19408 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:48.986806 19408 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:48.987608 19408 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:48.989930 19408 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:48.996637 19607 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 2.188s	user 0.001s	sys 0.019s
I20260709 13:18:48.998452 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1911) took 1978 ms. Trace:
I20260709 13:18:48.998798 16595 rpcz_store.cc:276] 0709 13:18:47.020272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.206078 (+185806us) service_pool.cc:224] Handling call
0709 13:18:47.206446 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.219872 (+ 13426us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:47.219899 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.219937 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:47.219972 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:48.997759 (+1777787us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54002184d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:48.997841 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:48.998153 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3663,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":12646,"spinlock_wait_cycles":30128384}
I20260709 13:18:49.013716 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1225) took 1699 ms. Trace:
I20260709 13:18:49.014233 16669 rpcz_store.cc:276] 0709 13:18:47.313950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.373406 (+ 59456us) service_pool.cc:224] Handling call
0709 13:18:47.373681 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.537542 (+163861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.537588 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.560968 (+ 23380us) log.cc:844] Serialized 26 byte log entry
0709 13:18:47.750161 (+189193us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:47.750212 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.750943 (+   731us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:49.013083 (+1262140us) raft_consensus.cc:1685] finished
0709 13:18:49.013126 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.013433 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11598,"raft.queue_time_us":84386,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":18447,"thread_start_us":305267,"threads_started":2,"wal-append.queue_time_us":183972,"wal-append.run_cpu_time_us":1826,"wal-append.run_wall_time_us":1014184}
I20260709 13:18:49.032511 16740 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:49.034065 19600 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.035377 18996 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:49.035280 19600 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.035943 18996 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.037467 18996 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.039631 18996 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:49.043072 16667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 299) took 1900 ms (client timeout 1875 ms). Trace:
W20260709 13:18:49.043852 16667 rpcz_store.cc:269] 0709 13:18:47.142008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.142134 (+   126us) service_pool.cc:224] Handling call
0709 13:18:49.042496 (+1900362us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b540009e554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.042758 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23121792}
I20260709 13:18:49.045475 19600 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.046293 19600 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.047096 19600 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.049618 19600 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.042745 19652 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.052805 19652 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.055155 19652 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.065667 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.068099 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 535) took 1462 ms. Trace:
I20260709 13:18:49.068509 17412 rpcz_store.cc:276] 0709 13:18:47.605711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.605923 (+   212us) service_pool.cc:224] Handling call
0709 13:18:49.065284 (+1459361us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b5800198c98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:18:49.067988 (+  2704us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2300672}
I20260709 13:18:49.071138 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:49.066718 19289 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.725s	user 0.006s	sys 0.021s
I20260709 13:18:49.071971 17412 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:49.071897 19289 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.073379 16664 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:49.073519 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.074816 19408 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:49.075219 19408 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.075278 19289 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.075778 19408 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.075979 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1848) took 2054 ms. Trace:
I20260709 13:18:49.076462 16668 rpcz_store.cc:276] 0709 13:18:47.021042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.021211 (+   169us) service_pool.cc:224] Handling call
0709 13:18:47.021586 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.720870 (+699284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.075440 (+1354570us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b540023da54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.075776 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606619904}
I20260709 13:18:49.078972 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2470) took 2027 ms. Trace:
I20260709 13:18:49.079406 16744 rpcz_store.cc:276] 0709 13:18:47.051422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.051523 (+   101us) service_pool.cc:224] Handling call
0709 13:18:47.051761 (+   238us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.189192 (+137431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.189226 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.212759 (+ 23533us) log.cc:844] Serialized 26 byte log entry
0709 13:18:47.309362 (+ 96603us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:47.309418 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.309511 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:47.895699 (+586188us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:49.078365 (+1182666us) raft_consensus.cc:1685] finished
0709 13:18:49.078396 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.078684 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118224,"raft.queue_time_us":161313,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":19547,"spinlock_wait_cycles":6722816,"thread_start_us":87153,"threads_started":2,"wal-append.queue_time_us":67250,"wal-append.run_cpu_time_us":6696,"wal-append.run_wall_time_us":1591312}
I20260709 13:18:49.086093 17413 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:49.087188 17412 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:49.089214 19510 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:49.092458 16738 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:49.086124 19408 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.093731 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 369) took 1498 ms. Trace:
I20260709 13:18:49.094134 16738 rpcz_store.cc:276] 0709 13:18:47.595387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.658328 (+ 62941us) service_pool.cc:224] Handling call
0709 13:18:49.093524 (+1435196us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400049a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.093680 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3181452032}
I20260709 13:18:49.096566 17419 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.097251 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 330) took 2105 ms. Trace:
I20260709 13:18:49.097658 17413 rpcz_store.cc:276] 0709 13:18:46.992254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.994330 (+  2076us) service_pool.cc:224] Handling call
0709 13:18:46.994663 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.086719 (+2092056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.096933 (+ 10214us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b540016c454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.097151 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4808632192}
I20260709 13:18:49.105445 19408 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.107482 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:49.108266 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.117400 19350 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.121095 17413 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:49.122331 17413 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 5
I20260709 13:18:49.123456 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.122635 17413 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.6
I20260709 13:18:49.124739 17413 raft_consensus.cc:2917] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Aborting config change with OpId 7.6: Aborted: Op aborted by new leader
I20260709 13:18:49.124993 19613 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.125224 17413 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 8 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:49.129336 19408 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:49.129942 19408 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.131260 19408 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.133670 19597 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:49.146782 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:49.149271 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:49.151216 19510 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:49.152303 19510 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:49.152895 19510 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.167721 19448 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.170517 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:49.173265 19456 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.174000 19456 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:49.154249 19510 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.174783 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.176141 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2506) took 2201 ms. Trace:
I20260709 13:18:49.176538 16592 rpcz_store.cc:276] 0709 13:18:46.974354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.974624 (+   270us) service_pool.cc:224] Handling call
0709 13:18:47.020127 (+ 45503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.026963 (+  6836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.026991 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:47.027410 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 13:18:47.090604 (+ 63194us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:47.090658 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:47.090745 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:48.966054 (+1875309us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4800002958. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:18:49.175593 (+209539us) raft_consensus.cc:1685] finished
0709 13:18:49.175612 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.175916 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6452,"raft.queue_time_us":7279,"raft.run_cpu_time_us":20051,"raft.run_wall_time_us":1922954,"spinlock_wait_cycles":6198144,"thread_start_us":61966,"threads_started":1,"wal-append.queue_time_us":192134,"wal-append.run_cpu_time_us":25069,"wal-append.run_wall_time_us":1517842}
I20260709 13:18:49.182193 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:49.190047 19448 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:49.197405 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:49.200380 19549 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.210788 19448 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:49.212157 19572 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:49.212556 19572 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.213184 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 334) took 2186 ms. Trace:
I20260709 13:18:49.213595 19572 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.213568 17410 rpcz_store.cc:276] 0709 13:18:47.031428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.038904 (+  7476us) service_pool.cc:224] Handling call
0709 13:18:47.039229 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:47.216041 (+176812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.212592 (+1996551us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540030cd54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.212986 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314944256}
I20260709 13:18:49.221699 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:49.222590 17414 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:49.227939 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2479) took 2059 ms. Trace:
I20260709 13:18:49.228358 16737 rpcz_store.cc:276] 0709 13:18:47.168195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.168367 (+   172us) service_pool.cc:224] Handling call
0709 13:18:47.168726 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.282402 (+113676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:47.282433 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:47.282478 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:47.282508 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.227474 (+1944966us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003ee9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.227541 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:49.227737 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":260944768}
W20260709 13:18:49.234264 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.238750 19456 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.239761 19456 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:49.240250 16587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 232) took 3826 ms (client timeout 1549 ms). Trace:
W20260709 13:18:49.240655 16587 rpcz_store.cc:269] 0709 13:18:45.413779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.413916 (+   137us) service_pool.cc:224] Handling call
0709 13:18:49.239894 (+3825978us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b540010cf54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.240044 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976497408}
I20260709 13:18:49.241865 19456 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.242916 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2485) took 3509 ms. Trace:
I20260709 13:18:49.243299 16590 rpcz_store.cc:276] 0709 13:18:45.733686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:45.736989 (+  3303us) service_pool.cc:224] Handling call
0709 13:18:45.737313 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.146926 (+1409613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.242429 (+2095503us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400052654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.242727 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3139032448}
I20260709 13:18:49.234305 17414 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:49.252750 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.255256 19408 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:49.256433 19408 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.257591 19408 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.260356 16737 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:49.263877 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2503) took 2332 ms. Trace:
I20260709 13:18:49.264335 16596 rpcz_store.cc:276] 0709 13:18:46.931665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.931887 (+   222us) service_pool.cc:224] Handling call
0709 13:18:46.932208 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:47.011842 (+ 79634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.263335 (+2251493us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.263651 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112153984}
W20260709 13:18:49.268445 19635 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 1.988s	user 0.000s	sys 0.017s
I20260709 13:18:49.279328 19468 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:18:49.284919 17415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 339) took 2249 ms (client timeout 1918 ms). Trace:
W20260709 13:18:49.285873 17415 rpcz_store.cc:269] 0709 13:18:47.035205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.035341 (+   136us) service_pool.cc:224] Handling call
0709 13:18:49.284340 (+2248999us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54005938d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.284593 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32058112}
W20260709 13:18:49.287743 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08002462e0 after lost signal to thread 16528
I20260709 13:18:49.288682 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.290203 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.291270 19468 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:49.291739 19468 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.292148 19468 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:49.292253 19602 raft_consensus.cc:2764] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:18:49.291944 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:49.296069 19602 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:49.303611 19358 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:49.304625 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49082.1us
I20260709 13:18:49.305423 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:49.306501 19510 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.307588 19602 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:49.307298 19510 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.321257 19510 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.325891 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.331636 19468 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.332819 19522 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:49.333418 19522 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:49.334563 19468 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.345225 19554 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.350605 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.351208 19554 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.350350 19537 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.352308 19537 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.353615 19554 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.354099 19556 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.354352 19537 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.334686 19522 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.356009 19522 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:49.357044 19533 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.343864 19602 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:49.345295 19379 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.350307 19311 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.342141 19559 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:49.360258 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 193573us
I20260709 13:18:49.364979 19358 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.365164 19311 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:49.365954 19642 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:49.365643 19358 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:49.366423 19642 raft_consensus.cc:1068] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Unable to promote non-voter ba2df6a668144022a223e504ae571f21: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:18:49.366891 19143 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:49.367697 19311 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.368345 19379 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.368861 19358 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.369148 19559 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.370432 19379 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.371474 19559 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.372239 19554 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:49.373360 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:49.373199 19554 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.374051 17416 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.372236 19374 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.375522 19374 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.376833 19537 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.379352 19535 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.379976 19535 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.381878 19535 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.382133 19572 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:49.378204 16588 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:18:49.383950 19602 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.384747 19456 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.385084 19408 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.385460 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2472) took 2310 ms. Trace:
I20260709 13:18:49.385391 19456 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.385927 16740 rpcz_store.cc:276] 0709 13:18:47.075226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.075363 (+   137us) service_pool.cc:224] Handling call
0709 13:18:47.075640 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.033217 (+1957577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.385129 (+351912us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54002b93d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.385374 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4499233152}
I20260709 13:18:49.386381 16737 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:18:49.387187 16737 pending_rounds.cc:85] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 2
I20260709 13:18:49.387342 16739 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:49.387486 16737 pending_rounds.cc:107] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:18:49.387684 19456 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.387809 16737 raft_consensus.cc:2889] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:49.388245 19456 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.388908 19456 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.389004 19651 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.389926 19651 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.391294 19456 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.391990 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.392283 19651 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:49.392925 17420 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:49.393182 19408 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.395756 19408 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.396672 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.397503 16738 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:49.399798 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.400760 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.402191 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.402880 16737 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:49.403630 19537 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.405866 19537 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.406878 19554 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:49.407588 19310 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:18:49.408456 19537 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:49.409195 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44065us
I20260709 13:18:49.409047 19537 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:49.409780 16671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1214) took 2168 ms (client timeout 2077 ms). Trace:
I20260709 13:18:49.410027 19522 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.410789 19311 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.411101 19679 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.411453 19311 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.411759 19679 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.413376 17416 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:49.410069 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:49.389047 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2486) took 2091 ms. Trace:
I20260709 13:18:49.399897 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.413959 19679 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.414669 16739 rpcz_store.cc:276] 0709 13:18:47.298069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.299907 (+  1838us) service_pool.cc:224] Handling call
0709 13:18:49.387019 (+2087112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.387800 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.388862 (+  1062us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54002b93d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.388996 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4773612288}
I20260709 13:18:49.421825 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:49.427445 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64395.8us
I20260709 13:18:49.420557 16738 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:49.409000 19680 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:49.420560 19311 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.377691 19374 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:49.415010 16671 rpcz_store.cc:269] 0709 13:18:47.241420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.337144 (+ 95724us) service_pool.cc:224] Handling call
0709 13:18:49.409400 (+2072256us) spinlock_profiling.cc:243] Waited 61.8 ms on lock 0x7b540009ea54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.409608 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":142051584}
I20260709 13:18:49.434316 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.424606 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.433749 19543 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:49.435513 19379 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:49.436591 16561 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:49.438037 19680 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:49.414598 19522 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.444145 19379 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:49.439388 17409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 532) took 1869 ms (client timeout 1847 ms). Trace:
I20260709 13:18:49.441078 19543 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:49.456589 17374 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.549s (SENT)
I20260709 13:18:49.451099 19642 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:49.494014 19642 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.495236 19642 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.464056 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:49.464856 19582 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:49.491340 19379 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:49.491071 17409 rpcz_store.cc:269] 0709 13:18:47.567606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.568943 (+  1337us) service_pool.cc:224] Handling call
0709 13:18:49.435636 (+1866693us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:49.499532 16664 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 294) took 2377 ms (client timeout 2733 ms). Trace:
W20260709 13:18:49.500046 16664 rpcz_store.cc:269] 0709 13:18:47.121705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.123235 (+  1530us) service_pool.cc:224] Handling call
0709 13:18:49.499261 (+2376026us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400346f54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.499463 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4326648704}
I20260709 13:18:49.500563 19543 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.501878 19543 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:49.466032 19677 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:49.480716 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 241) took 2902 ms. Trace:
I20260709 13:18:49.504434 16559 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:49.504514 16588 rpcz_store.cc:276] 0709 13:18:46.578156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:46.578302 (+   146us) service_pool.cc:224] Handling call
0709 13:18:49.377601 (+2799299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:49.383240 (+  5639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:49.480438 (+ 97198us) spinlock_profiling.cc:243] Waited 2.8 s on lock 0x7b540010cf50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:49.480636 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6437000448}
I20260709 13:18:49.488607 19680 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:49.481568 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 562199us
I20260709 13:18:49.486248 19350 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:49.508374 19350 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.510545 19532 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.490119 19549 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.490353 19381 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.512639 19582 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.512547 19532 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.513010 19381 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.513266 19582 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.515067 19381 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.515233 19678 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:49.515421 19311 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:49.515786 19678 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.515923 19311 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:49.516289 19678 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:49.491302 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800228fa0 after lost signal to thread 16620
I20260709 13:18:49.517151 19311 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.512676 19549 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.510893 19553 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.518710 19311 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:49.519403 19553 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.520529 19549 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.521543 19553 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.522238 19530 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:49.511184 19538 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:49.523082 19651 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:18:49.511900 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 47944.8us
I20260709 13:18:49.510893 19350 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.511886 19546 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:49.517829 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800228fc0 after lost signal to thread 16621
I20260709 13:18:49.451877 17376 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.514787 19532 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:49.526008 16525 debug-util.cc:398] Leaking SignalData structure 0x7b080028c8e0 after lost signal to thread 16695
W20260709 13:18:49.526408 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003534e0 after lost signal to thread 16696
I20260709 13:18:49.526948 19546 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.493036 19268 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:49.528322 19522 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.528050 19268 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.529510 19546 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.530107 19268 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.523536 19651 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:49.523559 19538 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.531641 19651 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.532167 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.532845 19538 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.515245 19677 consensus_queue.cc:1243] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:49.532927 19651 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.533908 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.534595 16671 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:49.524241 16559 leader_election.cc:400] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:49.535089 16559 leader_election.cc:403] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:49.535188 19472 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:49.535655 19472 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:49.536173 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.536542 19472 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.537789 19472 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.537839 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.533044 16664 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:18:49.527514 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003d99a0 after lost signal to thread 16769
W20260709 13:18:49.538833 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800354d20 after lost signal to thread 16770
W20260709 13:18:49.540223 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003da580 after lost signal to thread 17446
W20260709 13:18:49.540544 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003dbaa0 after lost signal to thread 17447
W20260709 13:18:49.554299 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 114165us
W20260709 13:18:49.563150 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55621.5us
I20260709 13:18:49.563257 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.569522 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.570278 16665 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:49.574203 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.574759 19537 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.575515 19577 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:49.575440 19379 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.576122 19577 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:49.576578 19577 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:49.579197 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:49.581218 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 71050.9us
I20260709 13:18:49.582791 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.584219 16665 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:49.584910 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.580247 19379 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:49.586135 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92790.9us
I20260709 13:18:49.586715 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:49.568080 19677 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.590001 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.579360 19374 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.578893 19522 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:49.594420 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.592334 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.592062 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.593642 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:49.585512 19582 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.610956 19350 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:49.585421 19537 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:49.618038 19537 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.620225 19537 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.620411 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:49.597976 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.598698 19677 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.602311 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.603355 16737 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:49.608392 16587 raft_consensus.cc:2376] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6. Re-sending same reply.
I20260709 13:18:49.608798 16494 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 5, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 5 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:49.609963 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.621578 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.626160 19350 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:49.582734 19602 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:18:49.627583 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.631176 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.631939 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.630587 16494 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 4 to 6, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 6 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:49.632573 16587 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:49.633136 16494 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:49.636099 16494 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:49.638603 16494 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 4 to 9, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 9 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:49.641191 19677 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.641029 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.642817 16592 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:49.644644 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.645649 16668 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:49.629943 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.646500 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.647188 16662 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:49.647367 16667 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:49.648363 16739 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:49.648470 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.650314 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.651288 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.653201 16667 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:18:49.654819 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.658226 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.659754 16591 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:49.660156 19522 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.662194 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.662433 19522 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:49.663036 16596 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:49.663594 16595 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:49.664641 16664 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:49.666117 16665 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:18:49.668547 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.670753 16588 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:49.671710 16591 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:18:49.672020 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51742us
I20260709 13:18:49.673622 19553 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:49.675225 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.675945 16592 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:49.676577 19381 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:49.677130 19560 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:49.679018 19538 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:49.679435 19538 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:49.673653 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:49.680444 19538 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.681705 19553 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:49.681958 19538 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:49.674613 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.686401 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.689132 19651 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:18:49.689546 19651 raft_consensus.cc:1068] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Unable to promote non-voter ba2df6a668144022a223e504ae571f21: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:18:49.690115 19307 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:49.691107 19307 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:18:49.691637 19602 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:49.692045 19602 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:49.692893 19680 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.692970 19602 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.694244 16590 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:49.694501 19602 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:49.697400 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.634454 16666 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:49.689463 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.706306 16596 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:49.709594 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:49.713871 16739 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:49.715431 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:49.716087 16491 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 212) took 29565 ms (client timeout 15000 ms). Trace:
I20260709 13:18:49.716161 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:49.715751 16494 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 8, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:49.717525 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.717717 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.717427 16494 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:49.719009 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.719664 16742 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:49.719125 16494 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 6, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:49.720697 16494 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:49.721503 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.722170 16740 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:49.721946 16494 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 4 to 7, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:49.723527 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:49.723708 16494 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 6, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 6 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:49.725550 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:49.725445 16494 catalog_manager.cc:5697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:49.726806 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:49.727052 16494 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 6, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 6 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:49.729066 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.736304 16587 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:49.734400 19522 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:49.737723 16589 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:49.738134 16589 pending_rounds.cc:85] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 7
I20260709 13:18:49.738360 16589 pending_rounds.cc:107] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:49.738556 19560 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:49.739362 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.738703 16589 raft_consensus.cc:2889] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:49.740491 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.741088 16745 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:18:49.737855 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.741933 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:49.742308 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.737748 19522 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:49.743129 19522 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:49.746349 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.746819 19677 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:49.747012 16740 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:49.747205 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.747486 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.747867 16590 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:49.748571 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.748644 19582 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:49.749341 19677 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:49.749465 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.749714 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.750137 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:49.755533 16588 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:49.750370 19677 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:49.750952 19311 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:49.751003 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:49.751547 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.758483 16592 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:49.752833 19572 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:49.753468 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.753556 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.738361 16667 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:49.757274 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.757407 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.754897 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.763397 16740 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:49.765779 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:49.766626 16744 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:49.761061 16589 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:49.768466 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:49.769347 19307 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:49.769881 19307 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:49.771027 19307 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.778719 16739 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:49.781016 19382 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:49.794301 16590 raft_consensus.cc:2410] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:49.739372 19560 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:49.801786 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.805572 16666 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:49.803574 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.816588 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:49.806897 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.807523 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.826787 16744 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 3.
I20260709 13:18:49.761263 16494 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 4, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), pending config changed. New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:18:49.818907 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 68114us
W20260709 13:18:49.819044 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57917.8us
I20260709 13:18:49.818112 16745 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
W20260709 13:18:49.821869 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.806604 19350 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:49.824618 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.825291 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:49.759822 19311 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:49.823333 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.832448 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.733613 19678 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.836428 16736 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:49.800894 19546 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:49.803599 19680 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:49.848073 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:49.850291 17415 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:18:49.716553 16491 rpcz_store.cc:269] 0709 13:18:20.152768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:20.195020 (+ 42252us) service_pool.cc:224] Handling call
0709 13:18:20.348702 (+153682us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:18:49.715846 (+29367144us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:45.471985 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:45.700795 (+228810us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:45.700834 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:18:45.700855 (+    21us) tablet.cc:662] Decoding operations
0709 13:18:45.701909 (+  1054us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:45.701975 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:45.702005 (+    30us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:18:45.787910 (+ 85905us) tablet.cc:701] Row locks acquired
0709 13:18:45.787937 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:18:45.853135 (+ 65198us) write_op.cc:270] Start()
0709 13:18:45.884859 (+ 31724us) write_op.cc:276] Timestamp: P: 1783603125847097 usec, L: 0
0709 13:18:45.884914 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 13:18:45.927635 (+ 42721us) log.cc:844] Serialized 4342 byte log entry
0709 13:18:46.881826 (+954191us) op_driver.cc:464] REPLICATION: finished
0709 13:18:47.086954 (+205128us) write_op.cc:301] APPLY: starting
0709 13:18:47.090772 (+  3818us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:47.208548 (+117776us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:47.218821 (+ 10273us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:47.535345 (+316524us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:47.603375 (+ 68030us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:18:47.607035 (+  3660us) write_op.cc:312] APPLY: finished
0709 13:18:47.739497 (+132462us) log.cc:844] Serialized 83 byte log entry
0709 13:18:47.746013 (+  6516us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:47.755218 (+  9205us) write_op.cc:454] Released schema lock
0709 13:18:47.768414 (+ 13196us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:47.783815 (+ 15401us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:47.785679 (+  1864us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2444230,"reported_tablets":32,"spinlock_wait_cycles":28159872,"child_traces":[["op",{"apply.queue_time_us":161841,"apply.run_cpu_time_us":6793,"apply.run_wall_time_us":704122,"num_ops":11,"prepare.queue_time_us":854630,"prepare.run_cpu_time_us":15766,"prepare.run_wall_time_us":950658,"raft.queue_time_us":239155,"raft.run_cpu_time_us":13609,"raft.run_wall_time_us":128896,"replication_time_us":971270,"thread_start_us":1455601,"threads_started":4,"wal-append.queue_time_us":571914,"wal-append.run_cpu_time_us":17000,"wal-append.run_wall_time_us":2195185}]]}
I20260709 13:18:49.855006 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:49.846068 16494 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:49.836227 19350 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:49.860611 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.858162 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.872751 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:49.819396 19572 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.890353 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:49.890918 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.890426 19577 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:49.895440 16667 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:49.894325 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:49.891950 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.859371 16494 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 6, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), now has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:49.851294 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:49.897699 16669 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:49.893821 16595 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
I20260709 13:18:49.900573 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:49.847760 19382 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:18:49.863898 19350 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:49.932889 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.985623 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:49.986967 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:50.045706 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 172385us
I20260709 13:18:49.899986 16494 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:18:50.063261 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.863445 17413 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:50.072167 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:50.067520 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:49.880198 19537 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:18:50.076452 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 183770us
I20260709 13:18:50.077164 16662 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:50.082634 16668 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:50.080932 16494 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:18:49.954493 19143 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:50.086603 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 279637us
I20260709 13:18:50.122372 16666 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:50.120988 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.132198 16741 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:50.136729 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:49.986411 19560 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.093658 19143 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:50.133725 16494 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 4, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:50.139099 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.121165 19268 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:50.133641 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4830651489999127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1728s for a response
W20260709 13:18:50.147011 16561 connection.cc:256] RPC call timeout handler was delayed by -1.46120245599991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 13:18:50.147310 16561 connection.cc:256] RPC call timeout handler was delayed by -1.3551900499999192s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 13:18:50.147645 16561 connection.cc:256] RPC call timeout handler was delayed by -1.3489651489999233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 13:18:50.147957 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2605024559998128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:18:50.148232 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2200024559997473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:18:50.148546 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0761618349997661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:18:50.148852 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0497850129997914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
I20260709 13:18:50.145778 16743 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:50.131352 19374 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8.
I20260709 13:18:50.152244 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:50.153154 16666 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:50.156247 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 284641us
I20260709 13:18:50.088255 19311 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.146443 19143 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:50.167634 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.168263 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:50.145831 19268 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.127274 19537 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.164515 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.165957 16494 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:50.177273 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.177336 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.182852 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.183948 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.171875 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:50.180166 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 91774.2us
I20260709 13:18:50.194234 16593 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:50.179050 19692 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:18:50.188593 16494 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 4 to 7, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:50.199021 19470 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:50.207152 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:50.207484 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.209889 19537 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:50.172749 19268 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:50.228415 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.229400 16590 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:50.234164 16662 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2.
I20260709 13:18:50.224771 16494 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 4, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:18:50.185032 19678 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:50.238229 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.244540 16587 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:50.240604 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.235489 16494 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 1 to 3, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 3 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:50.222993 19470 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:50.247843 19470 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:50.243467 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.292779 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.293778 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:50.294721 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.297621 16590 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:50.231251 19532 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:50.303949 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:50.304459 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 108588us
W20260709 13:18:50.298476 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.308856 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:50.304104 19685 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.387s	user 0.001s	sys 0.000s
I20260709 13:18:50.321391 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:50.323400 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.324247 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.323385 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.324827 16668 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:50.324748 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.325699 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.326577 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.326651 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.327059 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.327490 16671 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:50.327672 16670 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:50.327878 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.328267 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.328711 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.329020 16670 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:50.329823 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.330560 16664 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:18:50.330929 16664 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 5
I20260709 13:18:50.330850 19677 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:50.331279 16664 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 7.6
I20260709 13:18:50.331604 16664 raft_consensus.cc:2917] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Aborting config change with OpId 7.6: Aborted: Op aborted by new leader
I20260709 13:18:50.331735 19677 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:50.331998 16664 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:50.311398 19538 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:50.327796 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.317138 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.316771 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.335667 16593 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:50.314198 19678 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:50.342254 16559 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.339922 16590 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:50.336081 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 266748us
I20260709 13:18:50.332175 19677 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.358266 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:50.360671 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.338342 19538 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.369098 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.355412 19680 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:50.373023 16587 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:50.356781 19577 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:50.381215 16594 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:50.363626 19677 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:50.368099 19538 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:50.389609 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.390986 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.392122 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.392732 16590 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:50.393777 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.394976 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.396365 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.397583 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.398442 16590 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:50.399375 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.458715 16590 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:50.377983 19680 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:50.373040 19678 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:18:50.409693 19522 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.646s	user 0.003s	sys 0.009s
I20260709 13:18:50.501498 19696 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.512807 19696 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:50.502113 19680 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:50.398763 19143 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:50.520848 19143 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:50.404981 19350 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:50.502519 19678 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:50.522913 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.523288 19678 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:50.523589 16738 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:50.524598 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.525542 19268 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.522923 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:50.405771 19602 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.001s
I20260709 13:18:50.526520 16663 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:50.384642 19572 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:50.527022 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.533138 16588 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:50.526970 19522 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:50.535231 17409 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:50.536433 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:50.536600 19546 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:50.537062 19522 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:50.537093 19546 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:50.538254 17413 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:50.537395 16736 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:50.511071 19470 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:50.455180 19472 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.005s	sys 0.019s
I20260709 13:18:50.414362 19577 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.117s
I20260709 13:18:50.545838 19268 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:50.500649 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.549160 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.549849 16589 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:50.551743 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.552090 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.552685 16588 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:18:50.541877 19472 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.006s	sys 0.019s
I20260709 13:18:50.560443 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.560799 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:50.562059 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.565404 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.567180 16671 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:50.567788 17416 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:50.563107 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:50.570583 16592 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.007s	sys 0.009s
I20260709 13:18:50.570456 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.572279 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.572651 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.585883 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:50.559875 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.574486 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.577687 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.560439 17409 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:50.579309 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.589735 16743 raft_consensus.cc:2410] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:18:50.581208 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:50.581876 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 261989us
W20260709 13:18:50.582916 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.570564 19350 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:50.599287 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 241131us
W20260709 13:18:50.575735 19646 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.005s	sys 0.028s
I20260709 13:18:50.566939 19530 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:18:50.577744 19532 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:50.609872 16588 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:18:50.597066 16592 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.007s	sys 0.009s
W20260709 13:18:50.600812 16636 connection.cc:256] RPC call timeout handler was delayed by -1.2862932979999187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
I20260709 13:18:50.599575 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.560806 19702 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:18:50.627153 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:50.604563 19646 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.005s	sys 0.028s
I20260709 13:18:50.592378 16742 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:50.582640 19696 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:50.629748 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 128158us
I20260709 13:18:50.576349 19538 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.586041 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:50.636104 16595 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:50.611132 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:50.637385 16589 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:50.636507 16590 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
W20260709 13:18:50.634471 16636 connection.cc:256] RPC call timeout handler was delayed by -1.2243768209998507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
I20260709 13:18:50.626484 19560 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.605561 19572 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:18:50.643851 16588 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:18:50.636348 17413 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:18:50.637173 19350 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:50.645275 19374 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:50.661592 19698 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.454s	user 0.001s	sys 0.003s
I20260709 13:18:50.653375 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.666013 19470 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:50.668637 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.657547 17413 pending_rounds.cc:85] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 7
I20260709 13:18:50.673265 16588 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:50.686199 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.685736 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.691970 16588 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:18:50.693004 19530 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:50.693617 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:50.694276 16745 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:50.695165 19675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.000s	sys 0.016s
W20260709 13:18:50.695762 19693 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.496s	user 0.003s	sys 0.000s
I20260709 13:18:50.697049 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2519) took 1095 ms. Trace:
I20260709 13:18:50.697414 16594 rpcz_store.cc:276] 0709 13:18:49.601778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.601950 (+   172us) service_pool.cc:224] Handling call
0709 13:18:50.696884 (+1094934us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54001faed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:50.696978 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1243105024}
W20260709 13:18:50.698566 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45705.6us
I20260709 13:18:50.699421 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:50.663262 19350 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.701000 16591 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:18:50.675086 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.674063 17413 pending_rounds.cc:107] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:50.721203 16589 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:18:50.669298 19374 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:50.725623 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50610.9us
I20260709 13:18:50.730382 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:50.731405 19522 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:50.731801 19522 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.732151 19522 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:50.732185 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:50.735430 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 80857.8us
I20260709 13:18:50.735195 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:50.738265 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:50.739162 17416 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:50.738279 16742 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:50.735549 19143 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:50.720721 19472 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:50.722043 19350 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:50.746629 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.728111 17413 raft_consensus.cc:2889] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:50.766294 19582 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:50.787492 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.788803 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.772912 19311 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:50.736232 19374 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:50.781188 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 44177us
I20260709 13:18:50.783406 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.742245 19143 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:50.687247 19560 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.700009 19532 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:50.797452 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.700564 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:50.798087 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.848258 16666 raft_consensus.cc:606] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:50.798274 19713 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:50.798807 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:50.798175 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.799484 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.801155 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.857059 17412 raft_consensus.cc:2410] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:18:50.801816 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55264.2us
I20260709 13:18:50.846468 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.802337 19715 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:50.803354 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.804095 19717 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:50.804981 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:50.865329 16667 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:50.805698 19549 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.747184 19702 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.100s
W20260709 13:18:50.881732 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 149621us
I20260709 13:18:50.882468 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:50.849959 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:50.886359 16741 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:50.867919 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.851821 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.866559 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.870924 17420 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:50.808015 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:50.879640 19577 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.847945 19143 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:50.894538 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 44687us
I20260709 13:18:50.896253 17410 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 4: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:18:50.906127 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 157100us
I20260709 13:18:50.906356 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:50.906703 16664 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:50.855872 19713 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.909192 16669 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:50.864692 19715 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:50.872740 19717 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:50.919960 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:50.920759 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:50.923808 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:50.822793 19677 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:50.899672 19702 consensus_queue.cc:1243] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:50.907600 19642 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:50.916867 19713 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:50.965651 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:50.972580 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54144.4us
I20260709 13:18:50.921728 19715 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:50.933878 19538 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:50.923404 19717 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:50.983814 19530 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:50.983954 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64063.7us
I20260709 13:18:51.002403 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:50.969199 19734 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:50.920517 19651 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:18:50.899288 19577 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:50.993360 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:50.953363 19696 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.024487 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:50.996199 16587 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:50.974726 16662 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:50.920845 19549 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.011873 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:51.026475 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52982.2us
I20260709 13:18:51.026625 16669 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:50.978628 19582 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:51.032682 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:51.033213 16589 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:51.036473 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.036645 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.047180 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:51.049196 16670 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:51.003677 19530 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.036378 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:51.022126 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 82672.2us
I20260709 13:18:50.974301 19642 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.058012 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.058804 19617 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:51.059749 16662 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:18:51.060588 19732 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:18:51.060914 19734 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:51.055688 19530 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:51.064754 19546 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
W20260709 13:18:51.062572 19730 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:18:51.063738 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:51.065209 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.066625 16669 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 9.
I20260709 13:18:51.069378 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:51.035530 19582 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:51.061932 19718 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:18:51.078737 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.063803 19646 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:51.066330 19382 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.034808 19696 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.104384 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:51.108242 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.076447 19734 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:51.075395 19617 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:51.077246 19702 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.110430 19651 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.121537 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.086167 19582 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:51.169754 16671 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.112886 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.172889 16669 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:51.171448 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:51.110741 19472 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:51.115105 19646 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:51.086469 19546 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.120687 19549 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.112066 19741 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.114097 19470 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:51.154088 19680 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.623s	user 0.008s	sys 0.003s
W20260709 13:18:51.169266 19735 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.118s	user 0.001s	sys 0.002s
I20260709 13:18:51.115422 19743 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:51.174630 19730 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.002s
I20260709 13:18:51.122545 19642 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:51.123651 19577 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.120239 19677 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.132315 19696 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:51.176342 19678 consensus_meta.cc:287] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
W20260709 13:18:51.178018 19744 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:18:51.113806 19734 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:51.194561 16669 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:51.179966 19739 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.200659 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:51.201737 19731 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.106s	user 0.003s	sys 0.003s
I20260709 13:18:51.191123 19749 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.180198 19560 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.210940 19532 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.120409 19382 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.117096 19268 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:18:51.194986 19572 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:51.268764 16669 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:51.176849 19651 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.251538 16670 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.251698 16662 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:51.204603 19743 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.122416 19522 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.254556 19749 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:51.255579 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 147404us
I20260709 13:18:51.195216 19680 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:51.263945 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 151148us
W20260709 13:18:51.266760 16742 consensus_meta.cc:287] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.002s
I20260709 13:18:51.183043 19702 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.241225 19678 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.275388 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.254945 19677 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:51.276273 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:51.297715 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.258224 19382 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:51.306900 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.308142 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.267678 19572 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:51.313491 16588 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.228814 19741 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.318223 16590 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:51.276098 19743 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.326741 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.276305 19749 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.321022 16662 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:18:51.300899 16742 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:51.306365 19677 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.271626 19742 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:18:51.335299 16662 pending_rounds.cc:85] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 7
W20260709 13:18:51.317610 19733 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.057s	user 0.003s	sys 0.000s
I20260709 13:18:51.299180 19755 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:51.341100 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.319175 19572 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.282711 19532 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.352782 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.342571 16662 pending_rounds.cc:107] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:18:51.361758 16662 raft_consensus.cc:2889] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:51.391517 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.392675 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:51.341810 19740 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:18:51.398257 16670 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4.
W20260709 13:18:51.399284 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 138423us
I20260709 13:18:51.340572 19677 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:51.274919 19617 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.454463 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:51.355700 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:51.446626 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.734s (SENT)
I20260709 13:18:51.439677 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.462625 16669 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:51.301496 19472 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 13:18:51.453652 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:51.455368 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:51.283044 19646 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.456475 16588 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:18:51.470222 19374 consensus_meta.cc:287] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.579s	user 0.006s	sys 0.009s
W20260709 13:18:51.447355 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 139235us
I20260709 13:18:51.456211 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:51.471949 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:51.474231 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 146399us
W20260709 13:18:51.460953 19350 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.638s	user 0.005s	sys 0.005s
I20260709 13:18:51.459707 19546 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.479025 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.480892 16671 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:51.483501 19759 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.484179 19759 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.484465 19760 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:51.483556 19522 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.485530 19522 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:18:51.485884 19522 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.486277 19522 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:51.486518 19759 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.486840 19651 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:51.491714 19753 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:18:51.493716 19741 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.405321 19755 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.495656 19532 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.496465 19755 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.496753 19702 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:51.496801 19522 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.497730 19755 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.498181 19755 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.498597 19755 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.499475 19522 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.479962 19350 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.502135 19350 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.501994 19717 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.470655 19374 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.497402 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2544) took 1747 ms. Trace:
I20260709 13:18:51.504352 19374 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.505293 16589 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:51.503695 16592 rpcz_store.cc:276] 0709 13:18:49.749804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:49.759088 (+  9284us) service_pool.cc:224] Handling call
0709 13:18:49.799323 (+ 40235us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:49.841017 (+ 41694us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:50.637288 (+796271us) log.cc:844] Serialized 14 byte log entry
0709 13:18:50.857805 (+220517us) log.cc:844] Serialized 14 byte log entry
0709 13:18:50.865102 (+  7297us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:50.894485 (+ 29383us) log.cc:844] Serialized 26 byte log entry
0709 13:18:50.894611 (+   126us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:50.894665 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.894728 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.083668 (+188940us) spinlock_profiling.cc:243] Waited 4.53 ms on lock 0x7b18002a6ee0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:51.497178 (+413510us) raft_consensus.cc:1685] finished
0709 13:18:51.497200 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.497343 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24438,"prepare.queue_time_us":391459,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":87,"raft.queue_time_us":50832,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":21542,"spinlock_wait_cycles":10421504,"thread_start_us":923098,"threads_started":2,"wal-append.queue_time_us":338375}
I20260709 13:18:51.506435 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:51.479146 19757 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:51.506659 19755 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.507344 19757 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.507947 19757 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.508878 19755 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.510740 19754 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:18:51.511345 19754 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.511487 19745 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:51.511755 19754 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:18:51.512042 19745 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.473158 19537 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:51.532397 19582 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:51.533342 19713 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.003s
I20260709 13:18:51.514338 19742 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:18:51.476676 19472 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.534957 16591 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:51.536079 19717 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.536445 19702 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:51.537388 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1941) took 1321 ms. Trace:
W20260709 13:18:51.538072 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55978.7us
I20260709 13:18:51.557523 16595 rpcz_store.cc:276] 0709 13:18:50.216298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.637171 (+420873us) service_pool.cc:224] Handling call
0709 13:18:50.649756 (+ 12585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:50.654486 (+  4730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.654546 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:50.664428 (+  9882us) log.cc:844] Serialized 26 byte log entry
0709 13:18:50.980137 (+315709us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:50.980191 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.980276 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.537168 (+556892us) raft_consensus.cc:1685] finished
0709 13:18:51.537192 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.537333 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1680,"raft.queue_time_us":1830,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":14185,"thread_start_us":256128,"threads_started":1,"wal-append.queue_time_us":373173}
W20260709 13:18:51.538136 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 59206.8us
I20260709 13:18:51.562326 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:51.542352 19715 consensus_meta.cc:287] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.516s	user 0.002s	sys 0.007s
I20260709 13:18:51.562000 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.566500 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.538715 19537 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.567952 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.561843 19745 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.570597 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:51.570739 19734 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.566187 19754 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.572654 19743 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.575968 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:51.576579 19728 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.609s	user 0.001s	sys 0.009s
I20260709 13:18:51.580075 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1924) took 1417 ms. Trace:
I20260709 13:18:51.578219 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:51.597625 16592 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.597316 16587 rpcz_store.cc:276] 0709 13:18:50.170439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.608194 (+437755us) service_pool.cc:224] Handling call
0709 13:18:50.608505 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.025637 (+417132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.579876 (+554239us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540010de54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:51.580023 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890979072}
I20260709 13:18:51.584383 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:18:51.593840 17374 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.875s (SENT)
I20260709 13:18:51.599476 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:51.594826 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 323431us
I20260709 13:18:51.579769 19755 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.602741 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:51.594682 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:51.579457 19537 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:51.557333 19468 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:18:51.577531 19717 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.561985 19472 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.624563 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:51.615510 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:51.626955 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.613713 16593 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:51.593313 19760 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.553637 19713 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.638888 19602 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.639319 16667 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:51.639657 19582 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.640153 19729 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:18:51.640556 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:51.642269 19530 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.001s
I20260709 13:18:51.642721 19530 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.643373 16595 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
W20260709 13:18:51.643806 19750 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 13:18:51.644511 19546 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.644973 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.645570 16595 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:51.645715 19546 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.647596 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1944) took 1430 ms. Trace:
I20260709 13:18:51.647958 16590 rpcz_store.cc:276] 0709 13:18:50.217075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.642952 (+425877us) service_pool.cc:224] Handling call
0709 13:18:50.643250 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.328177 (+684927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.647391 (+319214us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400339fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:51.647541 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541757696}
W20260709 13:18:51.649632 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49413.7us
I20260709 13:18:51.604228 19382 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:18:51.610455 19522 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:51.592973 19678 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.605270 19374 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:51.593528 19680 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.653771 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.615118 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:51.624181 19532 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:51.657277 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.577435 19761 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:18:51.623759 19717 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.595839 19741 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:18:51.663385 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2512) took 1090 ms. Trace:
W20260709 13:18:51.667075 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55401us
I20260709 13:18:51.677059 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:51.676026 16740 rpcz_store.cc:276] 0709 13:18:50.573277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.574078 (+   801us) service_pool.cc:224] Handling call
0709 13:18:50.574415 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:50.578846 (+  4431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:50.578901 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:50.594450 (+ 15549us) log.cc:844] Serialized 26 byte log entry
0709 13:18:50.755450 (+161000us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:50.763643 (+  8193us) log.cc:844] Serialized 14 byte log entry
0709 13:18:50.786105 (+ 22462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:50.786178 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.575891 (+789713us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:51.651708 (+ 75817us) raft_consensus.cc:1685] finished
0709 13:18:51.663156 (+ 11448us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:51.663311 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55027,"prepare.queue_time_us":44691,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":22585,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":22063,"spinlock_wait_cycles":54364416,"thread_start_us":120326,"threads_started":2,"wal-append.queue_time_us":266084}
I20260709 13:18:51.678913 19755 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:51.657054 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:51.667876 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:51.668087 17413 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:51.680222 17412 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:18:51.668320 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:51.667577 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:51.577730 19715 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.662998 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:51.607764 19675 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:18:51.672967 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.672011 19582 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:51.673666 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.662371 19522 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:51.694847 19522 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:51.695349 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.696180 16662 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:51.697409 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.676769 19717 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.665948 19374 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.687863 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:51.664757 19677 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.690704 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.734434 16671 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9.
I20260709 13:18:51.691187 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:51.691500 19766 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:18:51.691900 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.692005 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.692816 16664 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:51.632807 19749 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.665486 19729 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:18:51.738209 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.680595 19468 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:18:51.700937 19556 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:51.684463 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:51.745784 19572 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:51.694394 19311 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.851s	user 0.008s	sys 0.009s
I20260709 13:18:51.683166 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:51.674186 19646 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.689931 19715 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.730453 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:51.730635 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45469.1us
I20260709 13:18:51.683991 19759 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:51.732040 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44298.3us
I20260709 13:18:51.732664 19532 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.732724 19743 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.767802 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.770298 19577 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:51.747083 19530 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:51.771415 19754 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:51.772013 19143 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.855s	user 0.005s	sys 0.005s
I20260709 13:18:51.772460 19734 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.773698 19472 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.774328 16594 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:51.775126 19702 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.775504 19677 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.733421 19374 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:51.775825 19761 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:18:51.776479 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 192181us
I20260709 13:18:51.817356 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.777879 16596 raft_consensus.cc:2410] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:51.779335 16592 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:18:51.786123 19470 consensus_meta.cc:287] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.592s	user 0.015s	sys 0.002s
I20260709 13:18:51.778472 19755 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:51.822484 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.823729 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.785848 19532 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:51.792598 19577 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:51.793671 19582 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:51.817947 19765 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.837670 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.840025 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.842317 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.843225 16662 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:51.850407 19767 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.851030 19767 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.851344 19572 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.852849 19749 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:51.853377 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:51.854231 19755 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.857054 19646 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:51.858094 19715 raft_consensus.cc:993] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:18:51.858569 19715 raft_consensus.cc:1081] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:18:51.819298 19472 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:51.816691 19553 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:51.784761 19757 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.816505 19761 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:51.835739 19532 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.808682 19143 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.832799 19470 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:18:51.864905 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47615.2us
I20260709 13:18:51.868662 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.874483 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.862067 19752 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:51.880205 16595 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:51.881175 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:51.882313 16665 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:51.883677 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 210066us
I20260709 13:18:51.884423 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:51.885466 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:51.886870 19725 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.887457 19725 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:51.887862 19725 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.876549 19646 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:51.890394 16708 leader_election.cc:400] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 9.
I20260709 13:18:51.890723 16708 leader_election.cc:403] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:51.866838 19553 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:51.894999 19769 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:51.896961 19560 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:51.862003 19765 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.895128 19768 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:18:51.898736 16708 connection.cc:256] RPC call timeout handler was delayed by -1.2854038440000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 13:18:51.919310 16708 connection.cc:256] RPC call timeout handler was delayed by -1.2818038440000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
W20260709 13:18:51.919890 16708 connection.cc:256] RPC call timeout handler was delayed by -1.2602038439999887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 13:18:51.920372 16708 connection.cc:256] RPC call timeout handler was delayed by -1.2400869240000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
W20260709 13:18:51.921782 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.641s (SENT) [suppressed 35 similar messages]
I20260709 13:18:51.931898 19757 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.901391 19678 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:51.936441 19678 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.936836 19678 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.940559 19717 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.943070 16590 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:51.919778 19577 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.944221 19577 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:51.946110 19522 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:51.948514 19522 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.923094 19472 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:51.956998 16670 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:18:51.958496 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.963956 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.894783 19760 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:51.897403 19560 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:51.966316 16662 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:18:51.967466 16666 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:51.970239 16559 leader_election.cc:400] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9.
I20260709 13:18:51.970569 16559 leader_election.cc:403] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:18:51.951022 19717 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:51.976437 16739 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:51.977257 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.962050 19757 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:51.976083 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:51.980414 16666 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:51.977244 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.922713 19582 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:51.982103 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.982378 19560 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:51.984156 19748 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.866s	user 0.002s	sys 0.013s
W20260709 13:18:51.984980 19782 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:18:51.992162 19765 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:51.975301 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:51.997774 16662 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:51.973167 19472 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9.
I20260709 13:18:52.017074 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:52.027567 19787 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:18:52.028820 19758 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:18:52.029297 19758 raft_consensus.cc:1068] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Unable to promote non-voter ba2df6a668144022a223e504ae571f21: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:18:52.029716 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 506) took 1244 ms. Trace:
I20260709 13:18:51.886921 19761 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.030066 17416 rpcz_store.cc:276] 0709 13:18:50.785337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.786049 (+   712us) service_pool.cc:224] Handling call
0709 13:18:50.852839 (+ 66790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:50.936706 (+ 83867us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:50.936757 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:50.960807 (+ 24050us) log.cc:844] Serialized 26 byte log entry
0709 13:18:51.030929 (+ 70122us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:51.030967 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:51.031030 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:51.983728 (+952698us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b240031b2a8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327c97d9c4 00007f327c97bacb 00007f327cb36dc9 00007f327cb36a55 00007f327cb3609d 00007f327f76810e 00007f327f77f88c 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221
0709 13:18:52.029404 (+ 45676us) raft_consensus.cc:1685] finished
0709 13:18:52.029427 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.029639 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14089,"raft.queue_time_us":22761,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":28798,"spinlock_wait_cycles":41770368,"thread_start_us":27212,"threads_started":1,"wal-append.queue_time_us":144982}
I20260709 13:18:52.031441 19761 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.035377 19708 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:52.035771 19708 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:52.035841 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:51.994158 19757 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.036669 19708 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.037899 19708 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:52.037946 19143 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.037087 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:52.039774 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:52.042505 19537 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.043480 19768 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 9.
I20260709 13:18:52.035991 19769 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:52.044741 19537 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.045646 19560 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:52.044814 16587 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:52.045536 19532 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.049686 19532 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.050930 16668 raft_consensus.cc:606] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.051380 16668 raft_consensus.cc:614] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:52.051730 19717 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.052852 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:52.052204 19717 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.068900 19374 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.069769 19717 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.071168 16589 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:52.072300 19678 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.074365 19553 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.076997 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:52.077699 16664 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:52.078454 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:52.079850 19602 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:52.080260 19602 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:52.080363 19678 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:52.081269 19602 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.083371 16664 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:52.085095 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 109852us
W20260709 13:18:52.058548 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 136773us
I20260709 13:18:52.062395 19725 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.058830 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:52.104375 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0456019170001127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 13:18:52.115134 16708 connection.cc:256] RPC call timeout handler was delayed by -1.0190038440000535s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2042s for a response
W20260709 13:18:52.116132 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.117343 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.140388 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.141172 16588 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:52.142249 16588 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:52.143380 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.082810 19602 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.144119 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.144193 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.146721 16591 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:52.135732 19717 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:52.148927 16665 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:52.149155 16588 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:52.142371 19769 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.158489 16593 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9.
I20260709 13:18:52.159494 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.159922 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.160506 16665 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:52.161100 16588 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:52.163220 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:52.163919 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.163807 16588 raft_consensus.cc:2376] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 4. Re-sending same reply.
I20260709 13:18:52.167594 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.168926 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.169693 16588 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:52.171591 16740 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:52.142549 19765 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:18:52.183246 19725 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:52.183524 16591 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:18:52.154850 19374 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.174394 19522 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:52.184818 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.184710 19522 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.186131 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.187033 19768 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:52.187343 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.187604 19768 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:52.188167 19768 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.183665 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.191733 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.192374 16588 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:52.193216 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.193923 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:52.194798 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.195614 16588 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:52.196672 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.197664 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.199331 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.200024 16588 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 9.
I20260709 13:18:52.200697 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.201862 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:52.198311 19761 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.204866 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.205925 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.208148 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.202315 16588 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:52.198590 19760 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.198554 19725 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:52.211941 19642 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.212881 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.213562 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.206799 19572 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.207401 19758 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:52.217922 19758 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:52.218288 19758 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.202729 19708 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:18:52.219745 19708 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:52.220129 19708 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:52.220999 19708 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.222185 19708 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:52.222229 19268 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:52.208938 19761 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.237979 19767 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.243597 19725 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:52.244108 19725 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:52.244342 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.244524 19725 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:52.246464 19758 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.249723 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.250104 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2561) took 1045 ms. Trace:
I20260709 13:18:52.250593 16587 rpcz_store.cc:276] 0709 13:18:51.204470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.598878 (+394408us) service_pool.cc:224] Handling call
0709 13:18:51.599283 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.047232 (+447949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.249934 (+202702us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54003b74d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:52.250060 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024219008}
I20260709 13:18:52.253597 19522 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.254314 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.253657 19761 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:52.255164 19761 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:52.255509 19761 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.256065 19760 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:52.256613 19760 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:52.258273 16596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1951) took 2039 ms (client timeout 1588 ms). Trace:
W20260709 13:18:52.258819 16596 rpcz_store.cc:269] 0709 13:18:50.218699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.266784 (+ 48085us) service_pool.cc:224] Handling call
0709 13:18:52.258005 (+1991221us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b540023bed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:52.258180 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3387079552}
I20260709 13:18:52.259233 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.260248 16744 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:52.260505 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.258759 19760 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:52.261654 19760 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:52.262056 19760 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:52.262441 19760 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.262614 16743 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:18:52.262475 19789 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:52.263168 16743 pending_rounds.cc:85] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 2
I20260709 13:18:52.263324 19789 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:52.263579 16743 pending_rounds.cc:107] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:18:52.263948 16743 raft_consensus.cc:2889] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:52.264364 19789 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.266023 19789 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:52.266165 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2560) took 1084 ms. Trace:
I20260709 13:18:52.267620 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 301) took 1099 ms. Trace:
I20260709 13:18:52.268088 16667 rpcz_store.cc:276] 0709 13:18:51.167895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.168060 (+   165us) service_pool.cc:224] Handling call
0709 13:18:51.168401 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.639706 (+471305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.267334 (+627628us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400002154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:52.267545 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082573184}
I20260709 13:18:52.272534 19522 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:52.273440 19522 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:52.273756 19522 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:52.274561 19522 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.275693 19522 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:52.276085 19757 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.278036 19713 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:52.279539 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.282270 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.266737 16594 rpcz_store.cc:276] 0709 13:18:51.181923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:51.258494 (+ 76571us) service_pool.cc:224] Handling call
0709 13:18:51.258791 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:51.807397 (+548606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.265887 (+458490us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400157a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:52.266094 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185236224}
I20260709 13:18:52.290261 16744 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:18:52.292419 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.294023 19572 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:52.294205 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.294389 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.296283 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:52.297972 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.299451 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 561) took 1449 ms. Trace:
I20260709 13:18:52.301851 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.299747 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:52.249028 19769 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.313721 16743 raft_consensus.cc:2368] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9.
I20260709 13:18:52.299817 17420 rpcz_store.cc:276] 0709 13:18:50.850244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:50.915156 (+ 64912us) service_pool.cc:224] Handling call
0709 13:18:50.915455 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:51.071767 (+156312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:51.071795 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:51.105253 (+ 33458us) log.cc:844] Serialized 26 byte log entry
0709 13:18:51.465127 (+359874us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:51.465180 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:52.248503 (+783323us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400573d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:52.248556 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:52.248601 (+    45us) raft_consensus.cc:1685] finished
0709 13:18:52.248614 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:52.299381 (+ 50767us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24036,"raft.queue_time_us":22102,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":8343,"spinlock_wait_cycles":317883776,"thread_start_us":331461,"threads_started":1,"wal-append.queue_time_us":383531}
I20260709 13:18:52.278496 19760 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.297124 19709 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:52.316720 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.316851 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.318380 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.318344 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:52.321259 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:52.322017 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.323730 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.324129 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.324404 16740 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.324564 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.325209 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.325579 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.326131 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.326552 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.323781 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.322157 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:52.327150 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.327824 16665 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:52.328075 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.327246 19709 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.324678 16736 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:52.327112 16737 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:52.328898 19642 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.329687 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.330119 19642 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.330809 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.331032 19739 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:52.331655 19739 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.328773 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.332484 16742 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:52.332430 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.335150 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.335785 16737 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:52.336638 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.337137 16737 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.328325 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.342144 16741 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:52.347430 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:52.348146 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:52.344354 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:52.352341 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.352591 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:52.353258 16743 raft_consensus.cc:2410] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:52.354084 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.354574 16743 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:52.297813 19764 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:18:52.353556 16663 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:52.332134 19642 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:52.360395 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.361210 16743 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:52.348771 19381 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.362584 19716 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:52.341223 19572 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:18:52.343072 19549 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.314174 19553 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:18:52.341109 19538 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:52.313154 19713 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:18:52.341461 19702 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:52.375955 19702 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:52.363992 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.370223 19789 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.356194 19530 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:52.355465 19696 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:52.383116 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.384265 19739 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.388767 16740 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:52.390172 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.394551 16736 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:52.395421 16744 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:52.397882 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.377297 19702 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.401274 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.402011 16740 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:52.377461 19538 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:52.403906 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:52.406741 19797 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:18:52.381371 19789 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.407569 19538 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.409444 19789 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:52.381466 19758 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:52.381304 19696 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:52.418486 16742 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:52.422302 16740 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:52.429790 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.430117 16736 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:52.430923 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.430765 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.431990 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.441156 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.443528 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.432708 16745 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:52.453987 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.454528 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.416728 19350 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:52.412431 19768 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.456151 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.457000 16588 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:52.400137 19642 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:18:52.475389 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.475771 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.478828 16745 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:52.453823 19522 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.455181 19767 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.487124 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.457221 19350 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:52.424173 19761 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:52.489180 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45945.2us
W20260709 13:18:52.490103 17374 leader_election.cc:336] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.046s (SENT)
I20260709 13:18:52.492154 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.470288 19757 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.495736 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:52.455137 19789 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:18:52.499323 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.494486 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.496498 16671 raft_consensus.cc:606] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.510520 16671 raft_consensus.cc:614] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:52.509621 16745 raft_consensus.cc:2410] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:18:52.511356 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.512436 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.513399 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.514747 16738 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:52.481495 19374 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:52.517135 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:52.516870 16739 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:52.518986 19781 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.516773 19765 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:18:52.516855 19717 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:18:52.525835 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:52.527310 19546 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:52.527704 19546 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:52.528085 19546 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:52.528713 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.531478 19268 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:52.508417 19143 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.535041 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:52.535421 19374 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.537581 19374 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:52.538476 19796 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:18:52.539558 16745 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:52.540169 19709 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:52.497120 19577 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:18:52.541927 19577 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:52.542271 19577 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:52.544031 16737 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:52.491008 19752 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:52.546056 16743 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:52.495672 19701 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:52.551201 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:52.550125 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 163675us
I20260709 13:18:52.555783 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.561739 16671 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:52.515410 19767 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:18:52.562606 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44056.8us
I20260709 13:18:52.564422 16745 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:52.566715 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.568572 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:52.569844 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:52.571110 16561 leader_election.cc:400] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9.
I20260709 13:18:52.571879 16736 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:52.576668 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:52.572196 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:52.575330 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.580021 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:52.579989 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:52.581593 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.583601 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.565387 19769 raft_consensus.cc:2764] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:52.592085 16736 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:18:52.585773 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 71562.4us
I20260709 13:18:52.576530 19522 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:52.577812 19374 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:52.575073 19717 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:52.597344 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:52.566489 19757 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:52.604518 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.509775 19696 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.575987 19530 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:18:52.583500 19767 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.610651 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.619573 16745 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:52.585312 19582 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:52.639146 19582 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.639578 19582 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:52.639906 19768 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:52.643177 19760 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:52.645040 16738 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:52.578671 19572 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:18:52.646790 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:52.648056 16738 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:52.649149 16742 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:18:52.649505 16742 pending_rounds.cc:85] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:18:52.649727 16742 pending_rounds.cc:107] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:18:52.649953 16742 raft_consensus.cc:2889] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:52.650148 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.651155 16737 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:52.654358 16742 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:52.572893 19739 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:18:52.573145 19758 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:18:52.585983 19752 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:52.632223 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:52.605165 19800 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:18:52.697952 16590 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:52.635816 16739 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:52.649132 16738 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:52.658778 16744 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:18:52.660149 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 78881.9us
I20260709 13:18:52.661144 16594 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:52.585944 19781 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:52.602046 19734 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:52.667348 16742 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:52.602991 19522 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:52.606357 19374 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:52.609460 19789 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:52.608508 19717 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:52.628130 19678 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:52.692389 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.693562 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:52.695323 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 98010.7us
I20260709 13:18:52.696161 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:52.666445 19802 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:18:52.703375 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.735051 16595 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:52.584887 19350 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.606460 19701 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 13:18:52.720156 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:52.720161 16589 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:52.634972 19769 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.089s
I20260709 13:18:52.654069 19757 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:18:52.579831 19577 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:52.733088 19717 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:52.671310 16745 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:18:52.610276 19804 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 13:18:52.730568 16587 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:52.745697 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:52.729914 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:52.746537 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 135959us
W20260709 13:18:52.747876 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:52.726811 19522 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:52.723960 19374 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:52.728495 16596 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:52.720230 19809 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:18:52.761649 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 61072.9us
I20260709 13:18:52.761840 16745 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:52.768332 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:52.704216 19470 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:18:52.787398 16587 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:52.739884 19752 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:52.704788 19538 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:18:52.702930 19803 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.086s
I20260709 13:18:52.784197 16745 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:52.785140 16592 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:52.734151 19268 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:18:52.725993 19617 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:52.724261 19734 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.743461 19716 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:18:52.796630 16596 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:52.824721 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 122925us
I20260709 13:18:52.812707 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:52.754745 19764 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:18:52.779642 19761 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:52.825892 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:52.757086 19546 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:52.814079 19789 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:52.783421 19715 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:18:52.816418 19809 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:52.784941 19646 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:18:52.847159 16595 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:52.832321 19781 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:52.881295 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55515us
I20260709 13:18:52.886022 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:52.890024 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:52.858574 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 120049us
I20260709 13:18:52.817329 19713 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:18:52.812954 19758 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:18:52.852039 19769 consensus_queue.cc:1243] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:52.937171 19765 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:52.852726 19143 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:18:52.913522 19678 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:18:52.998725 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53819.4us
I20260709 13:18:52.852802 19382 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
W20260709 13:18:52.956863 19778 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:18:52.953454 19532 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:18:52.956118 19820 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:53.014437 19816 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 13:18:53.015127 19734 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.016216 19781 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.021343 19582 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.022400 16591 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:53.012171 19820 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:53.023401 19820 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.025732 19546 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.027148 19829 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:18:53.027498 19811 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:18:53.027575 19829 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.027815 17420 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:53.028105 19829 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.029289 19758 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:53.029721 19758 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:53.030588 19758 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.031881 19818 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:53.032086 19789 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.030680 19374 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:53.049170 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.044001 16745 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:53.056751 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 288479us
I20260709 13:18:53.069074 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:53.004901 19821 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.031860 19758 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.007973 19268 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:18:53.082449 19268 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:53.082804 19268 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:52.999614 19752 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.083602 19268 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.084796 19268 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.029826 19809 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:53.091063 19702 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.666s	user 0.005s	sys 0.005s
I20260709 13:18:53.091504 19702 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:53.086673 19725 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.842s	user 0.010s	sys 0.000s
I20260709 13:18:53.076543 19821 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.075984 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.008777 19765 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:53.077907 19714 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:18:53.094794 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 336371us
I20260709 13:18:53.007689 19532 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.013098 19530 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:53.079525 16745 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.099750 16590 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
W20260709 13:18:53.100549 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 323976us
I20260709 13:18:53.075693 19824 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:52.999255 19617 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
W20260709 13:18:53.123739 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 393894us
I20260709 13:18:53.126888 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:53.128505 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:53.117630 19714 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.103720 19702 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.135947 16590 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:18:53.142148 19350 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:53.120560 19823 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:53.151937 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:53.146595 19804 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.148133 19828 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:53.104449 19725 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.153064 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.155025 16595 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:18:53.180284 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.186100 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.223948 19714 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.225751 19714 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:53.226265 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 108063us
I20260709 13:18:53.151368 19822 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.150013 19824 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:53.227579 19546 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.186036 19823 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:53.230762 16736 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:53.154294 19804 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.231216 19715 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:53.232389 19830 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:53.232693 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:53.235098 16741 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:53.230891 19834 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:53.161537 19828 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:53.156513 19716 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:18:53.152381 19803 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:18:53.197543 19617 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:18:53.232126 19381 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:53.130185 19678 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.236120 19813 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:53.234645 19546 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:18:53.243403 19767 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 13:18:53.257165 16595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:18:53.236042 19764 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:18:53.231348 19470 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:18:53.263056 16595 raft_consensus.cc:606] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:53.267459 16595 raft_consensus.cc:614] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:18:53.258917 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 368953us
I20260709 13:18:53.245255 19823 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:18:53.268785 16708 leader_election.cc:400] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 9.
W20260709 13:18:53.274588 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.275162 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:53.254282 19381 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.258569 19546 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.234441 19765 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.263824 19767 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:53.287961 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:53.272131 19374 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.236868 19532 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.347419 19824 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.348873 19832 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.349367 19832 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.349735 19832 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.296458 19812 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:53.351229 19812 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.355973 19822 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:53.358661 19807 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:53.265089 19813 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:53.359323 19814 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 13:18:53.250649 19830 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.360730 19725 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.228220 19821 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.294714 19764 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:53.367424 19821 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.367580 19764 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.368875 19764 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.289717 19835 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.371341 19835 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.265462 19717 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.293509 19530 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:18:53.297379 19702 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:53.261023 19833 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
W20260709 13:18:53.379051 19522 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.001s
I20260709 13:18:53.379031 19807 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:53.379669 19522 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.380035 19807 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:53.381615 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:53.382736 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:53.383172 19837 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:53.383934 19837 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:53.384114 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:53.384301 19837 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:53.384676 19837 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.384986 19642 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:18:53.385808 19530 consensus_queue.cc:1243] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:53.386303 19530 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:53.386593 19530 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:53.387360 16588 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:53.387418 19530 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.318353 19828 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.388473 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:53.388751 19530 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.389004 19828 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.389128 16736 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:53.347929 19836 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:18:53.390477 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:53.387144 19532 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:18:53.409348 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.410177 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.410638 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.434191 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:53.451025 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 321487us
I20260709 13:18:53.402009 19835 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:53.473850 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 241203us
I20260709 13:18:53.474694 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.477008 16595 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:18:53.477452 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 214155us
I20260709 13:18:53.486153 19752 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.486783 19752 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.487427 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:18:53.488060 16671 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.488384 16671 raft_consensus.cc:614] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:18:53.489131 19752 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.489744 19752 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:53.490180 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:18:53.468519 19809 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:53.490772 19809 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:53.491168 19809 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.492156 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:53.494225 16595 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:18:53.496336 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.447275 19701 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:18:53.504765 16590 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:53.506196 16587 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:53.508360 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:53.509784 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:53.515290 19803 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:53.517132 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:53.519464 17413 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:53.521831 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:53.547729 17415 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.532286 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:53.561808 17413 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:53.473035 19829 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:53.572944 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 182551us
I20260709 13:18:53.579535 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:53.576436 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:53.583246 16590 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:53.472885 19820 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.520961 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:53.490202 19752 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:53.595409 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 77133.2us
I20260709 13:18:53.600001 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.600859 16590 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:53.532234 19701 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 13:18:53.596364 19768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.004s	sys 0.028s
I20260709 13:18:53.619716 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:53.516448 19803 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.622557 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:53.630460 19768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.004s	sys 0.029s
I20260709 13:18:53.620041 19765 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:53.622426 19849 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:53.649935 19851 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 13:18:53.708398 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 107155us
I20260709 13:18:53.684434 19832 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.683713 19765 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.710296 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:53.699610 19842 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:18:53.696890 19617 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:53.682822 19861 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:18:53.700650 19858 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:53.723472 17412 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:53.725385 16588 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:53.688252 19849 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.716182 19546 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:53.732230 17409 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:53.716383 19752 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.717576 19862 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.721879 19856 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 13:18:53.744099 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 122284us
I20260709 13:18:53.749128 16588 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:53.722796 19858 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:53.727451 19678 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:53.739705 19829 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.738531 19837 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.722615 19617 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.760958 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:53.761447 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:53.765364 16736 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:53.767638 19865 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:53.767887 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.768095 19865 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:53.768278 19546 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.768237 19858 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.769079 19865 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.770339 19865 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.757678 19820 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.771222 16589 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:53.750988 19862 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.772300 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:53.773247 19862 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.773509 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.775277 19854 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.775803 19854 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.777942 19854 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.781044 19849 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.782303 19765 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.791141 17415 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:53.792007 17413 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:53.792752 16589 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:53.794034 19832 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.769577 19858 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:53.764796 19859 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:18:53.811353 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.812069 16589 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:53.813154 16664 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:18:53.813303 16666 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
W20260709 13:18:53.813609 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.814649 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.816920 16671 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:53.758023 19678 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.822446 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:53.821285 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.823587 16670 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:53.825350 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.812718 19820 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:53.829751 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2579) took 1169 ms. Trace:
I20260709 13:18:53.830166 16591 rpcz_store.cc:276] 0709 13:18:52.660122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.660231 (+   109us) service_pool.cc:224] Handling call
0709 13:18:52.694435 (+ 34204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:53.022755 (+328320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:53.829508 (+806753us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b540023bed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:53.829666 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753843584}
I20260709 13:18:53.831459 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.833231 19617 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.819669 19803 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.840408 19678 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.842105 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.843447 19871 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:53.844048 19708 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.844463 19708 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:53.844769 19809 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.845546 19871 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:53.845342 19708 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.847546 17411 raft_consensus.cc:2410] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:53.842828 16590 raft_consensus.cc:2368] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:53.834395 19813 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.851507 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:53.855859 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:53.851809 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.857141 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.851912 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.820079 19553 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:53.860432 16666 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:53.861158 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.862715 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.864123 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.865077 16590 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:53.864921 16664 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:53.858331 19530 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:53.859835 19702 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:53.866640 19708 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.868979 19702 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:53.870011 19702 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.873502 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:53.874291 16590 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:53.876519 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.880282 16663 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:53.880678 16663 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.881397 16663 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.883699 19530 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:53.884263 19530 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.885593 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:53.886276 19530 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:53.886991 16671 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:53.895989 19530 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:53.896584 19530 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.899561 19530 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.906586 19582 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.909065 16587 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:53.910113 19769 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:18:53.860251 19553 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.913270 19553 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.914263 19553 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.917027 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.917804 16669 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.919071 19768 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:53.919447 19268 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:53.919644 19768 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.920037 19268 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.921096 19268 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.921952 19768 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:53.922530 19768 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:53.912123 19805 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.923422 19805 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.924106 19350 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:53.924531 19350 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:53.924306 19805 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.906788 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:53.925448 19350 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.926540 19350 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:53.927997 19812 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:53.931486 19572 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.932981 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:53.934834 16736 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:53.936017 16736 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:53.936712 16736 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.936854 19381 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.937964 16736 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:53.939118 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47218.6us
I20260709 13:18:53.941057 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:53.941747 16590 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.942411 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:53.947335 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:53.922626 19268 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:53.952263 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 169773us
I20260709 13:18:53.952580 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:53.953277 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:53.953933 16737 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.955130 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:53.956368 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:53.957093 16739 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:53.958214 16742 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:53.939028 19572 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:53.981778 19572 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.982770 19572 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:53.942929 19582 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:18:53.986071 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 565) took 1406 ms. Trace:
I20260709 13:18:53.970512 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:53.968036 16737 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:53.969867 19824 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:53.971308 16736 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:18:53.971426 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.971153 19834 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:18:53.972697 19803 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:53.973685 19808 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.230s	user 0.009s	sys 0.006s
I20260709 13:18:53.974007 19532 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:53.976694 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.977123 19849 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:53.977169 19617 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:53.977738 19851 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:18:53.978400 19546 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:53.980365 16738 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:53.953876 19822 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:53.986979 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.030076 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.986500 17420 rpcz_store.cc:276] 0709 13:18:52.579581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.579728 (+   147us) service_pool.cc:224] Handling call
0709 13:18:53.985860 (+1406132us) spinlock_profiling.cc:243] Waited 73 ms on lock 0x7b5400091854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:53.985997 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167854848}
I20260709 13:18:53.961602 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:53.988215 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:53.990633 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:53.996953 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.996243 16590 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:53.945189 19713 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:53.999224 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:54.000497 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.998108 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:53.912181 19769 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:53.941095 19768 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:53.980696 19577 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.029759 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:54.029517 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 268123us
I20260709 13:18:54.029547 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:54.025254 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:54.033931 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:54.053632 16736 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:54.035194 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:54.035929 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:53.995721 19758 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:54.038998 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:54.039816 19716 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.005s	sys 0.005s
I20260709 13:18:54.043383 16708 leader_election.cc:400] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:54.046211 19374 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.069895 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:54.049937 19717 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.011613 19849 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.053107 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:54.055224 16670 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:54.054790 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:54.074069 16736 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:54.011073 19803 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.012382 19752 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.052501 16665 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:53.953397 19765 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:18:54.078693 16739 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:54.026734 19834 consensus_queue.cc:1243] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:18:54.062839 16740 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 3. Current term is 4. Ops: []
I20260709 13:18:54.062182 16708 leader_election.cc:403] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:18:53.961985 19812 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:18:54.071614 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:54.088322 16737 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:54.031306 19822 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.075074 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:54.075114 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:54.076473 16590 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:18:54.080159 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:54.060446 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:54.061895 19716 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.005s	sys 0.005s
W20260709 13:18:54.062124 16744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1095) took 1750 ms (client timeout 1549 ms). Trace:
I20260709 13:18:54.086632 16739 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:18:54.091264 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48103.2us
I20260709 13:18:54.217533 16662 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:54.320262 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:54.321291 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:18:54.321709 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.322230 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.322840 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.323714 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:54.324376 19841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.000s	sys 0.018s
W20260709 13:18:54.324753 19841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.000s	sys 0.018s
I20260709 13:18:54.325254 19538 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.325654 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:54.326575 19836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.005s	sys 0.007s
I20260709 13:18:54.256980 16736 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:54.209460 16670 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:18:54.327761 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:54.327811 19725 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:18:54.328150 16741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1108) took 2001 ms (client timeout 1731 ms). Trace:
I20260709 13:18:54.328438 19725 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:54.328851 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:54.329064 19725 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:54.329301 16743 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:54.061699 19713 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:54.134687 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74332.6us
I20260709 13:18:54.332242 19713 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.333165 19717 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:54.334200 16671 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:54.334734 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.326987 19836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.883s	user 0.005s	sys 0.007s
I20260709 13:18:54.100250 19758 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:18:54.212889 19881 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:54.088620 19752 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:54.210599 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 135617us
W20260709 13:18:54.192719 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.081349 19835 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:54.080891 19374 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.213517 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:54.205106 19678 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.081910 19861 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:54.099316 16744 rpcz_store.cc:269] 0709 13:18:52.311961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.312153 (+   192us) service_pool.cc:224] Handling call
0709 13:18:54.061909 (+1749756us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b5400426154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:54.062051 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167037184}
I20260709 13:18:54.213229 19803 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.078188 19530 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:54.083178 19851 consensus_queue.cc:1243] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:54.077644 19849 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.107154 19822 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.219993 19833 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:54.470813 16744 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:54.265367 16666 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:54.217140 19820 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:54.252067 19809 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.247714 19522 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.254246 19837 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.249022 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2576) took 1591 ms. Trace:
I20260709 13:18:54.251106 19805 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.269825 19830 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.233995 16587 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: [5.4-8.5]
I20260709 13:18:54.226310 17410 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:54.210497 16737 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:54.513486 16737 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:54.128456 19824 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:54.275238 19855 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.656s	user 0.003s	sys 0.000s
I20260709 13:18:54.217306 16669 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:54.280136 19572 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:54.532649 19572 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.280339 19764 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:54.283255 19577 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:18:54.284678 19847 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.607s	user 0.002s	sys 0.000s
I20260709 13:18:54.533907 19885 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.287405 19642 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:54.534161 19577 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.534538 19885 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.288859 16595 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:54.298144 19891 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:54.454190 19861 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:54.539372 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.301791 19810 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:18:54.303094 19813 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:54.304420 19715 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:18:54.307878 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:54.308887 19582 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:54.313066 19857 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.628s	user 0.003s	sys 0.000s
I20260709 13:18:54.314693 16742 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:18:54.558074 16742 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:18:54.558468 16742 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:18:54.316973 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2539) took 1889 ms. Trace:
I20260709 13:18:54.244588 19854 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:54.330636 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2532) took 1905 ms. Trace:
I20260709 13:18:54.469633 19530 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.334419 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:54.558795 16742 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:18:54.166540 19646 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.338472 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:54.337694 16662 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:54.027333 19532 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:54.328564 16741 rpcz_store.cc:269] 0709 13:18:52.326336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.352007 (+ 25671us) service_pool.cc:224] Handling call
0709 13:18:54.327963 (+1975956us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b5400434254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:54.328087 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973040896}
I20260709 13:18:54.352170 19381 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:54.352872 19860 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:18:54.355824 19848 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.736s	user 0.003s	sys 0.001s
W20260709 13:18:54.359616 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.533921 19764 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:54.372037 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 376185us
I20260709 13:18:54.573904 16662 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:54.445739 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.262941 19757 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.475s	user 0.017s	sys 0.042s
I20260709 13:18:54.362354 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:54.368259 19821 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.493578 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:54.370380 19881 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.500926 16744 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:54.325724 19538 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.504685 16593 rpcz_store.cc:276] 0709 13:18:52.658068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.658203 (+   135us) service_pool.cc:224] Handling call
0709 13:18:52.693164 (+ 34961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:52.850972 (+157808us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:52.851016 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:52.851073 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:52.851123 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.248787 (+1397664us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400339ad4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:54.248830 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:54.248961 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362288384}
I20260709 13:18:54.515817 16587 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:18:54.590884 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.517161 16736 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:54.520617 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:54.523062 16596 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:54.267207 19838 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Committing config change with OpId 6.6: config changed from index 2 to 6, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:54.534586 19642 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.288738 19768 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:54.300122 19696 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.294019 19804 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.492136 19865 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.480841 19833 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:54.551151 16595 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:54.574688 19860 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:54.652216 16595 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:54.539466 19836 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.432024 19678 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:54.559088 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.560231 16742 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:18:54.335232 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.561161 16740 rpcz_store.cc:276] 0709 13:18:52.427862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.644486 (+216624us) service_pool.cc:224] Handling call
0709 13:18:54.062666 (+1418180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.087677 (+ 25011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.316774 (+229097us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400426150. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:54.316911 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3260797696}
I20260709 13:18:54.561686 16663 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:54.336073 16594 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:54.562114 16667 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:54.560252 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 225914us
I20260709 13:18:54.559222 16743 rpcz_store.cc:276] 0709 13:18:52.425611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:52.563722 (+138111us) service_pool.cc:224] Handling call
0709 13:18:54.329134 (+1765412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.329722 (+   588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.330436 (+   714us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b5400434250. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:54.330576 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4059154816}
W20260709 13:18:54.573820 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 235420us
I20260709 13:18:54.664894 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:54.579926 16739 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:54.576726 16710 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:54.552672 19813 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:54.552879 19715 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:18:54.600258 16745 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.043s	sys 0.078s
I20260709 13:18:54.608114 16671 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:54.571981 19522 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:18:54.579463 19757 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.799s	user 0.017s	sys 0.042s
W20260709 13:18:54.683740 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 107071us
I20260709 13:18:54.684157 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:54.686208 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.649214 16666 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:54.541595 19577 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.579070 19758 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:54.579226 19845 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.544028 19824 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.669960 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:54.673274 16745 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.052s	sys 0.087s
W20260709 13:18:54.674502 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.691440 16595 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:18:54.699661 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:54.701712 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:54.794302 17420 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:54.702067 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:54.706138 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:54.692574 19817 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.682s	user 0.019s	sys 0.006s
I20260709 13:18:54.552407 19891 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:54.736066 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.736608 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:54.577518 19582 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.664153 19833 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.728559 17415 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:18:54.739154 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:54.801443 17415 pending_rounds.cc:85] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 7
I20260709 13:18:54.801728 17415 pending_rounds.cc:107] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:18:54.801957 17415 raft_consensus.cc:2889] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:54.749547 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:54.727133 19572 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.757831 19715 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:54.706634 19143 consensus_queue.cc:1175] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" responder_term: 8 status { last_received { term: 7 index: 6 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:18:54.760056 19813 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:54.763571 19888 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 13:18:54.768074 19374 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.769225 19530 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:54.771032 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:54.696593 19861 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:54.820984 19582 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.773761 19860 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.696199 19885 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:54.692811 19522 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:54.775203 19807 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:18:54.593647 19821 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.682313 19757 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:54.739042 19837 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:54.777861 19532 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.673754 19268 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
W20260709 13:18:54.779623 19886 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:18:54.779788 19824 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.705229 19889 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.410s
I20260709 13:18:54.694445 19804 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:54.723954 19845 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.747707 17416 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Committing config change with OpId 6.6: config changed from index 2 to 6, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:54.581247 19646 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.581849 19810 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
W20260709 13:18:54.799162 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.908977 16591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.021s	sys 0.055s
W20260709 13:18:54.909349 16591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.021s	sys 0.055s
I20260709 13:18:54.799180 17410 raft_consensus.cc:2376] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9. Re-sending same reply.
I20260709 13:18:54.802474 17414 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:54.754086 16667 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:54.912115 16667 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:54.673313 19470 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 13:18:54.689929 19830 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:18:54.682672 19854 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.112s
W20260709 13:18:54.800976 19817 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.793s	user 0.019s	sys 0.006s
I20260709 13:18:54.666182 19865 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:54.790989 19765 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.781893 19717 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:18:54.805716 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.809883 16663 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:18:54.940186 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1299) took 1079 ms. Trace:
I20260709 13:18:54.940614 16663 rpcz_store.cc:276] 0709 13:18:53.860832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.875678 (+ 14846us) service_pool.cc:224] Handling call
0709 13:18:53.879370 (+  3692us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.561486 (+682116us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b58004f19d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:18:54.939792 (+378306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.940117 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050539264}
I20260709 13:18:54.811151 19715 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:18:54.794379 19678 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.792196 19892 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:54.942780 19892 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:54.943231 19892 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:54.681671 19836 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:54.947046 19549 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:18:54.805683 19752 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:54.807879 19572 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:54.738519 19764 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.819664 19813 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:54.903187 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.886631 19807 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:54.907794 19860 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:54.973655 19764 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:54.975854 19701 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:18:54.976392 19701 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:54.977491 19701 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:54.979633 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.980043 19821 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.981328 19891 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:54.982205 19757 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:54.983315 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.983922 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:54.985237 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:54.986630 19865 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:54.992051 19845 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.992916 19867 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:54.993451 19867 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:54.997779 19678 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:18:54.998541 19646 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:54.999578 19853 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:55.000021 19853 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:54.999699 19678 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:54.911837 19881 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:54.935158 19854 consensus_queue.cc:1243] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:18:54.951618 19538 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.016192 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1922) took 1019 ms. Trace:
I20260709 13:18:55.015017 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1121) took 1075 ms. Trace:
I20260709 13:18:55.016705 16671 rpcz_store.cc:276] 0709 13:18:53.996508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.996648 (+   140us) service_pool.cc:224] Handling call
0709 13:18:55.015988 (+1019340us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54001f7054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:55.016112 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693419392}
W20260709 13:18:55.012454 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 350460us
I20260709 13:18:55.017232 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:55.017866 16738 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:55.018103 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.014755 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1981) took 1464 ms. Trace:
W20260709 13:18:55.023190 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.018662 16590 rpcz_store.cc:276] 0709 13:18:53.550312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.107074 (+556762us) service_pool.cc:224] Handling call
0709 13:18:54.107437 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.219196 (+111759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.219240 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.219775 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.483163 (+263388us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:54.483202 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.483270 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.014503 (+531233us) raft_consensus.cc:1685] finished
0709 13:18:55.014531 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.014687 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1832,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":262856,"threads_started":1,"wal-append.queue_time_us":322123}
I20260709 13:18:55.016919 16745 rpcz_store.cc:276] 0709 13:18:53.939527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.940209 (+   682us) service_pool.cc:224] Handling call
0709 13:18:53.943206 (+  2997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:53.943586 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.943626 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:53.945123 (+  1497us) log.cc:844] Serialized 26 byte log entry
0709 13:18:54.038117 (+ 92994us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:54.735384 (+697267us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.013903 (+278519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.014002 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.014053 (+    51us) raft_consensus.cc:1685] finished
0709 13:18:55.014803 (+   750us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.014963 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17201,"prepare.queue_time_us":720867,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":148504,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":16885,"thread_start_us":960838,"threads_started":3,"wal-append.queue_time_us":149967}
I20260709 13:18:55.001397 19530 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:55.001951 19642 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.010965 19549 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.036926 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.037727 16671 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:54.952205 19765 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:54.956534 19572 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.043170 19549 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.032007 19470 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.029449 19817 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:55.025385 19701 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:55.050462 19725 consensus_meta.cc:287] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.721s	user 0.005s	sys 0.004s
W20260709 13:18:55.051373 19898 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:18:55.052309 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.079607 16596 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:54.994464 19867 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.054052 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.056437 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.082509 16593 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:55.059223 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.060446 19837 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.075992 19764 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.062914 19849 consensus_meta.cc:287] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.571s	user 0.004s	sys 0.005s
I20260709 13:18:55.062804 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:55.064285 16664 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:55.067132 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2595) took 1566 ms. Trace:
I20260709 13:18:55.067780 19865 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.068439 19835 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.023375 19822 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.003s
I20260709 13:18:55.072364 16738 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:55.021958 19752 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.027040 19810 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:55.076992 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:55.077252 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.102082 16594 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:54.950933 19522 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.053457 16595 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:55.000959 19853 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.028884 19530 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:55.081616 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 397504us
I20260709 13:18:55.057788 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.079912 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.046628 19892 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.088258 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:55.089781 16670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.011s	sys 0.015s
I20260709 13:18:55.002315 19678 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
W20260709 13:18:55.079136 19713 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.030s	sys 0.030s
I20260709 13:18:55.091256 16592 rpcz_store.cc:276] 0709 13:18:53.501050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.501247 (+   197us) service_pool.cc:224] Handling call
0709 13:18:53.509161 (+  7914us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:53.509520 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:53.509550 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:54.377029 (+867479us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b2c000014c8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:18:54.377726 (+   697us) log.cc:844] Serialized 52 byte log entry
0709 13:18:54.815086 (+437360us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:54.824393 (+  9307us) log.cc:844] Serialized 14 byte log entry
0709 13:18:54.824721 (+   328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.824791 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.066811 (+242020us) raft_consensus.cc:1685] finished
0709 13:18:55.066936 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.067072 (+   136us) inbound_call.cc:177] Queueing success response
0709 13:18:55.069136 (+  2064us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b180030f040. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
Metrics: {"mutex_wait_us":39076,"prepare.queue_time_us":7266,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":513,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":17671,"spinlock_wait_cycles":10374912,"thread_start_us":319753,"threads_started":1,"wal-append.queue_time_us":583809}
I20260709 13:18:55.080869 19725 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.156201 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:55.156519 16738 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:55.134943 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 57553.9us
I20260709 13:18:55.157456 17410 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:55.108560 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:55.076107 19470 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.076025 19817 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.108384 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.092254 19867 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:55.157734 16588 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.012s	sys 0.033s
I20260709 13:18:55.136025 17414 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 8 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:18:55.158313 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49808.5us
W20260709 13:18:55.188452 16670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.012s	sys 0.016s
I20260709 13:18:55.214402 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:55.213681 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57537.5us
I20260709 13:18:55.215835 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:55.196578 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.089272 19837 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.159145 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:55.218818 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 59752.9us
I20260709 13:18:55.219053 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:55.191347 16588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.012s	sys 0.033s
I20260709 13:18:55.220659 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1985) took 1669 ms. Trace:
I20260709 13:18:55.222378 16743 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
W20260709 13:18:55.212052 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54591.8us
I20260709 13:18:55.092432 19764 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:55.116654 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 446734us
I20260709 13:18:55.226279 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:55.228060 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:18:55.192209 19696 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.542s	user 0.005s	sys 0.005s
I20260709 13:18:55.221997 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:55.124285 19854 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.102931 19642 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:55.084066 19807 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.234688 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1931) took 1235 ms. Trace:
I20260709 13:18:55.248785 16669 rpcz_store.cc:276] 0709 13:18:53.999790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.532731 (+532941us) service_pool.cc:224] Handling call
0709 13:18:54.533018 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.533203 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:54.948601 (+415398us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.234381 (+285780us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.234427 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:55.234472 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.234531 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.234635 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5009,"prepare.queue_time_us":109139,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4791,"raft.queue_time_us":14356,"thread_start_us":695518,"threads_started":2}
I20260709 13:18:55.107105 19143 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 13:18:55.254317 19530 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.255568 19530 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:55.258529 19805 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.716s	user 0.007s	sys 0.000s
I20260709 13:18:55.264814 19572 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.267052 16671 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
W20260709 13:18:55.271135 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49239.5us
I20260709 13:18:55.273250 19739 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.274022 19885 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:18:55.273910 19739 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.222345 16588 rpcz_store.cc:276] 0709 13:18:53.551524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.262952 (+711428us) service_pool.cc:224] Handling call
0709 13:18:54.263258 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.263467 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:18:54.372794 (+109327us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.220332 (+847538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.220389 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:55.220437 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.220502 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.220607 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52717,"prepare.queue_time_us":53820,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":136,"thread_start_us":903364,"threads_started":2}
I20260709 13:18:55.276757 19739 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.271271 19902 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:55.216295 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1930) took 1216 ms. Trace:
I20260709 13:18:55.155344 19702 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.214965 19853 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.158454 19381 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:55.159138 19538 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.155306 19717 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.124804 19836 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.244743 19803 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:55.109509 19830 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.125372 19845 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:55.100528 19865 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.190706 19758 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:55.187943 19713 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.035s	sys 0.035s
I20260709 13:18:55.250746 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.242795 19582 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:55.233007 19577 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:55.260246 19765 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.261730 19817 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.271154 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2600) took 1489 ms. Trace:
I20260709 13:18:55.103389 19835 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:55.280568 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.104691 19822 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.238026 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:55.278738 19902 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:55.237882 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:55.241204 19768 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:55.106238 19752 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.279740 16670 rpcz_store.cc:276] 0709 13:18:53.998697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.329695 (+330998us) service_pool.cc:224] Handling call
0709 13:18:54.329965 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:54.330137 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:54.471068 (+140931us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.215007 (+743939us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.215076 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:55.215133 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.215215 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.215371 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244349,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":5694,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11109,"spinlock_wait_cycles":40355584,"thread_start_us":883845,"threads_started":2,"wal-append.queue_time_us":782954}
I20260709 13:18:55.297354 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:55.228791 19907 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:55.248113 19821 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:18:55.249672 19854 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.337764 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1303) took 1266 ms. Trace:
I20260709 13:18:55.336658 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.300757 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1923) took 1304 ms. Trace:
I20260709 13:18:55.300138 19702 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.338702 19764 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.303565 19715 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:18:55.304373 19805 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.248724 19862 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.305944 16662 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:55.338824 16742 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:18:55.351222 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.373433 17415 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:55.109032 19810 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.254729 19549 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.301143 16663 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:18:55.315392 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.385902 16740 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:55.317577 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.386386 17410 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:55.318073 16743 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:18:55.318720 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:55.331724 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.387285 16738 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:55.321031 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.388243 16737 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:55.323128 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.324015 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.389210 17413 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:55.389432 16736 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:18:55.337224 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.337949 19851 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 13:18:55.337666 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 99006.3us
W20260709 13:18:55.396508 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.399920 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.310698 19717 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.279870 19903 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:55.309871 19838 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 13:18:54.556542 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:55.336292 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.309612 19538 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.095454 19849 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.358566 16664 rpcz_store.cc:276] 0709 13:18:53.996814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.999013 (+  2199us) service_pool.cc:224] Handling call
0709 13:18:55.300581 (+1301568us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540009db54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:55.300699 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2372556032}
I20260709 13:18:55.360117 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.431071 16670 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
W20260709 13:18:55.361244 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 63969.7us
I20260709 13:18:55.433480 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:55.344758 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:55.378271 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 140305us
W20260709 13:18:55.405097 19833 consensus_meta.cc:287] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.600s	user 0.002s	sys 0.008s
I20260709 13:18:55.435379 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:55.369167 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.367105 16669 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
W20260709 13:18:55.379678 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 55785.5us
I20260709 13:18:55.371387 19830 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.367339 19582 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:55.367488 19678 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:55.367292 19577 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:55.485857 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.486195 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:55.381384 16589 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1974) took 1833 ms (client timeout 1758 ms). Trace:
I20260709 13:18:55.309327 19803 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:55.489626 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.358139 16668 rpcz_store.cc:276] 0709 13:18:54.072016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.639961 (+567945us) service_pool.cc:224] Handling call
0709 13:18:54.666568 (+ 26607us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:54.667055 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.667096 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:54.681027 (+ 13931us) log.cc:844] Serialized 52 byte log entry
0709 13:18:54.956586 (+275559us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:54.956625 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.956691 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.337518 (+380827us) raft_consensus.cc:1685] finished
0709 13:18:55.337546 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.337702 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":907,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":20144,"thread_start_us":161939,"threads_started":1,"wal-append.queue_time_us":494826}
I20260709 13:18:55.390748 16741 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:55.490370 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.298648 19381 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:55.393096 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 161435us
I20260709 13:18:55.495363 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:55.401427 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:55.403298 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.301002 19642 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:55.485090 19835 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.500921 19642 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.359359 19715 consensus_queue.cc:1243] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:18:55.368079 19702 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.367930 19822 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.334285 19725 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.411552 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:55.466727 19765 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:55.508389 19538 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.508740 19765 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:55.509217 19765 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.419252 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1939) took 1082 ms. Trace:
I20260709 13:18:55.509675 19577 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.510226 16666 rpcz_store.cc:276] 0709 13:18:54.336925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.272139 (+935214us) service_pool.cc:224] Handling call
0709 13:18:55.272441 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.272619 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:18:55.272880 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:18:55.418842 (+145962us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:55.418918 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:55.418961 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.419030 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.419181 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14128,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":107,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":86,"thread_start_us":145553,"threads_started":1}
I20260709 13:18:55.369326 19892 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.359472 19902 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.400624 16587 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:18:55.424116 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.513127 17415 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:55.366587 19905 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:55.514026 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.514322 19905 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:55.514703 19905 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:55.515030 19752 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.515373 19381 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.515641 19911 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:55.516199 19911 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.494575 19803 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:55.518333 19911 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.514026 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.521119 17415 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:55.408627 19817 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:55.408416 19560 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.366279 19805 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.478011 19678 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:55.380363 19143 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:55.470355 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:18:55.471117 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.407786 19768 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:55.472585 16664 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:55.472517 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:55.475108 16665 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:55.475188 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1935) took 1474 ms. Trace:
I20260709 13:18:55.477380 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:55.478570 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.571559 16595 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:55.479602 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:55.385085 19862 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.369995 19758 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:55.576509 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 106265us
W20260709 13:18:55.467427 19913 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:55.365563 16591 rpcz_store.cc:276] 0709 13:18:53.782040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.331575 (+549535us) service_pool.cc:224] Handling call
0709 13:18:54.331882 (+   307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:54.332210 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.332239 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:54.332794 (+   555us) log.cc:844] Serialized 52 byte log entry
0709 13:18:54.909684 (+576890us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:54.909722 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:54.909798 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.270906 (+361108us) raft_consensus.cc:1685] finished
0709 13:18:55.270925 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.271098 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":157,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":77,"thread_start_us":576515,"threads_started":1,"wal-append.queue_time_us":767358}
W20260709 13:18:55.487231 16589 rpcz_store.cc:269] 0709 13:18:53.548097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:53.812599 (+264502us) service_pool.cc:224] Handling call
0709 13:18:55.381166 (+1568567us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54003395d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:55.381314 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454081920}
I20260709 13:18:55.458734 19757 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:55.465989 19904 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.466660 19854 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.459949 19907 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.618888 19538 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:55.619125 19907 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.619879 19538 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:55.620287 19538 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:55.621129 19830 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.621236 19907 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.611588 19572 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:55.622252 19572 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.623014 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.493961 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.498847 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.624254 19572 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.582384 19752 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.624801 19892 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.499961 16741 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:55.624964 19752 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.625331 19892 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.625970 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.535748 19549 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.521798 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:55.469295 19836 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.524453 16561 connection.cc:256] RPC call timeout handler was delayed by -1.3502306450000106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21480000000000002s for a response
W20260709 13:18:55.632084 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2953306449999218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
I20260709 13:18:55.559823 19837 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.463594 16744 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:55.465334 19833 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.537444 17416 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:55.564972 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.567452 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.543807 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1307) took 1331 ms. Trace:
I20260709 13:18:55.569479 16669 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:55.473624 19813 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:18:55.650585 16738 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:55.579018 19817 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:55.429702 19903 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:55.570045 16662 rpcz_store.cc:276] 0709 13:18:54.000723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.581155 (+580432us) service_pool.cc:224] Handling call
0709 13:18:54.581450 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.371863 (+790413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.474823 (+102960us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b540009d154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:55.475115 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665757568}
I20260709 13:18:55.612618 19470 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.615324 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:55.616564 19861 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:55.617224 19532 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:55.618305 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:55.627111 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.694585 16741 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:18:55.694815 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 76595.6us
I20260709 13:18:55.438326 16745 raft_consensus.cc:2410] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:18:55.621627 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.625056 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.695591 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:55.569284 19678 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:55.521162 19891 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.524303 19824 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:18:55.630790 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 109067us
I20260709 13:18:55.705933 16592 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:18:55.630324 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:55.706838 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76584.1us
I20260709 13:18:55.485132 19522 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.531718 19822 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:55.651134 16663 rpcz_store.cc:276] 0709 13:18:54.212506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.941775 (+729269us) service_pool.cc:224] Handling call
0709 13:18:55.300731 (+358956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.385544 (+ 84813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.543458 (+157914us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b540009db50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:55.543719 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824578432}
I20260709 13:18:55.651937 16740 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:55.758725 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1308) took 1540 ms. Trace:
I20260709 13:18:55.652999 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.576505 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:55.635360 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:55.654156 16710 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:55.613150 19757 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:55.625983 19915 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.118s	user 0.000s	sys 0.002s
W20260709 13:18:55.575172 19908 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:18:55.660238 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.557657 19764 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:55.660549 16662 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:18:55.794275 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.795152 16662 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:55.587742 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:55.661473 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.561676 19560 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.662638 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.800983 16589 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:55.696091 16743 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:55.699872 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:55.801903 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.802613 16663 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:55.700702 17419 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:55.637136 19881 raft_consensus.cc:2764] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:18:55.634572 19549 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.711278 16669 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:55.766470 19757 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:55.660594 19765 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.700629 19769 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:18:55.724061 19914 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 13:18:55.725241 19752 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.604615 19905 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.700583 19470 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:55.726817 19717 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:55.726958 19860 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.755s	user 0.008s	sys 0.003s
I20260709 13:18:55.634758 17409 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:18:55.735641 19923 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:18:55.621132 19538 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.707616 19891 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.737673 19903 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.738695 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.836941 19577 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.739008 19892 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.704748 19713 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:18:55.626966 19904 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.750991 19768 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.700835 19867 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:55.752975 19824 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:55.753786 19833 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.692656 19532 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:55.755143 19805 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.756276 19381 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.742741 19849 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:55.652501 19836 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.706513 19865 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.759161 16671 rpcz_store.cc:276] 0709 13:18:54.217876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.268975 (+1051099us) service_pool.cc:224] Handling call
0709 13:18:55.269387 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.269782 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.269815 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.270383 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.522974 (+252591us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:55.523010 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.523090 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.758500 (+235410us) raft_consensus.cc:1685] finished
0709 13:18:55.758520 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.758667 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27550,"raft.queue_time_us":18421,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4394,"thread_start_us":252168,"threads_started":1,"wal-append.queue_time_us":485042}
W20260709 13:18:55.761891 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 128905us
W20260709 13:18:55.764986 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46323.1us
I20260709 13:18:55.792128 16591 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:55.800191 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.761998 19861 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:18:55.805743 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 218094us
W20260709 13:18:55.938755 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.805862 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 106060us
I20260709 13:18:55.944895 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.632534 19582 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:55.813587 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 159499us
I20260709 13:18:55.816078 19549 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.950065 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.952647 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:55.702430 19803 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.818218 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1306) took 1723 ms. Trace:
W20260709 13:18:55.820200 19922 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:18:55.823330 19885 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:18:55.824007 16708 connection.cc:256] RPC call timeout handler was delayed by -1.013115665999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1572s for a response
W20260709 13:18:55.963341 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:55.963919 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.824438 19837 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:55.828948 16665 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1302) took 1758 ms (client timeout 1562 ms). Trace:
I20260709 13:18:55.837631 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.834081 16666 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:18:55.845237 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.800185 19810 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.762257 19522 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.832432 19925 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:18:55.871775 19905 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.766814 19822 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.812577 19715 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:55.836772 19813 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:55.804076 19764 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:55.843137 19470 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:55.850252 19560 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:55.897157 16739 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:55.897836 16742 pending_rounds.cc:85] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 3
I20260709 13:18:55.985780 16742 pending_rounds.cc:107] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:18:55.986178 16742 raft_consensus.cc:2889] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:55.839687 19817 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.895761 19824 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:55.928256 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:55.946203 19849 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:18:55.929759 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.889272 19577 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.945583 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:55.972774 19925 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:56.001030 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55526.5us
I20260709 13:18:56.001802 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:55.945991 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:18:56.003053 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57149.1us
I20260709 13:18:56.003698 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:55.946529 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:56.004529 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.936655 16662 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:56.005784 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.006251 16662 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:18:55.954675 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.006775 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.007258 16743 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:56.007737 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:55.933856 19854 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:55.933578 19807 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.925983 19717 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:55.951064 16592 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:18:55.961120 16667 rpcz_store.cc:276] 0709 13:18:54.094721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.914227 (+819506us) service_pool.cc:224] Handling call
0709 13:18:54.951824 (+ 37597us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:54.953538 (+  1714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:54.953567 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:54.954003 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.383057 (+429054us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:55.383095 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.383155 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:55.625461 (+242306us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:55.817894 (+192433us) raft_consensus.cc:1685] finished
0709 13:18:55.817922 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:55.818155 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1337,"raft.queue_time_us":1417,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":28497,"spinlock_wait_cycles":29596416,"thread_start_us":360124,"threads_started":1,"wal-append.queue_time_us":484941}
I20260709 13:18:55.964327 16593 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
W20260709 13:18:55.964383 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.965606 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:55.930383 19926 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.029758 19904 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:55.946379 19861 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.030583 19861 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:55.990072 19905 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:56.044037 19905 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.997905 16589 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:56.035476 19805 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:56.049104 19805 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:56.049515 19805 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:55.928815 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:55.998272 16590 raft_consensus.cc:2393] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:18:56.050668 19927 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:18:56.051337 16587 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.054471 19905 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.008309 16662 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:55.973150 19885 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:56.013363 19853 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.712s	user 0.007s	sys 0.005s
W20260709 13:18:56.056119 19933 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:18:56.065629 19885 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.066236 17414 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.013616 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.066720 19836 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.010179 19925 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:18:56.016462 19803 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.022114 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.067942 16596 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:56.025607 16671 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:55.994848 19470 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.029224 19926 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:55.967799 16665 rpcz_store.cc:269] 0709 13:18:54.070573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.093131 (+ 22558us) service_pool.cc:224] Handling call
0709 13:18:55.828652 (+1735521us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540001b654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:55.828844 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2542885632}
I20260709 13:18:56.044308 19940 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:18:56.074784 16589 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:18:55.996280 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:56.076017 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 79821.5us
I20260709 13:18:56.060221 16738 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:56.076786 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:56.072831 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:55.948936 19532 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.079488 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.080680 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:56.024621 19851 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.082947 19851 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.083357 19851 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.024744 19821 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:56.024430 19560 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.093755 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.094599 16592 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.089926 16671 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:18:56.090297 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:55.972865 19837 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:55.983564 19813 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.028250 19765 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.096946 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.120523 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.121409 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.097556 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.090655 19752 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:56.101497 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:56.136539 16666 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.101361 19861 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.096122 19572 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:56.102129 19715 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:56.103607 19926 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.152194 19926 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:56.152665 19926 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:56.092032 19646 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:18:56.093909 19560 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.109617 19862 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.074692 19881 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:56.115552 16596 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:56.126148 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.096580 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.127040 16744 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:56.166034 16744 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:56.127882 16740 raft_consensus.cc:1217] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 3.2->[4.3-9.5]   Dedup: 4.3->[6.4-9.5]
I20260709 13:18:56.167984 16740 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 3
I20260709 13:18:56.128605 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:56.140637 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 169403us
I20260709 13:18:56.149202 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.149411 16666 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:56.168309 16740 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:18:56.151104 19572 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:56.136271 19752 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.093657 19821 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.027287 19854 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.036866 19807 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.108192 19757 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.114373 19891 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.162945 16670 raft_consensus.cc:2368] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
W20260709 13:18:56.160745 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 82344.2us
I20260709 13:18:56.168715 16740 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:56.151510 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.184798 16667 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:56.184965 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.185557 16671 raft_consensus.cc:2376] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9. Re-sending same reply.
I20260709 13:18:56.185678 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.170037 16738 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:56.187728 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.188465 16738 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.189456 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.168617 16743 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.041842 19717 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.143067 19810 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.168274 16592 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.147678 19532 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:56.156920 19902 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.644s	user 0.010s	sys 0.004s
I20260709 13:18:56.149766 19758 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:56.143210 19702 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:56.173627 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:56.153719 19926 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.150471 19904 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:56.154717 19765 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:18:56.175818 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 99101us
I20260709 13:18:56.164018 19560 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.158231 19381 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.191771 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.186756 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:56.176757 19752 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:56.190127 16738 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:56.177577 19821 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:56.167127 16744 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.240183 16744 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.201709 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.164307 19470 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.206751 16666 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.208750 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:56.194320 19646 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.174064 19881 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.258765 16666 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:56.196388 19810 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:56.209378 19838 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.013s	sys 0.027s
I20260709 13:18:56.206035 19758 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.196213 19940 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:18:56.206357 19702 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:56.244341 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.251946 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.199877 19950 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:18:56.238147 19822 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.237605 19764 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.233769 19522 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.269238 16671 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:18:56.277834 19838 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.013s	sys 0.028s
I20260709 13:18:56.232450 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:56.269973 16744 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:18:56.271951 16666 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:56.292521 16744 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 6
I20260709 13:18:56.286242 16738 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:18:56.292325 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59998us
I20260709 13:18:56.287969 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.277575 19851 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.292230 16664 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:18:56.292877 16744 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.7
I20260709 13:18:56.232875 19904 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.301188 16666 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:56.297079 16738 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:18:56.298466 16744 raft_consensus.cc:2917] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Aborting config change with OpId 8.7: Aborted: Op aborted by new leader
I20260709 13:18:56.206761 19902 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.278278 19810 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.294476 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:56.300709 19905 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:56.283196 19758 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:56.287567 19702 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.314082 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:56.259987 19646 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.326320 19646 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.315332 16671 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:56.318077 16665 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:56.302573 16744 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:56.251968 19572 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.301139 19813 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.323450 17411 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:56.323350 17419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 527) took 1621 ms (client timeout 1946 ms). Trace:
W20260709 13:18:56.315281 19936 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.165s	user 0.001s	sys 0.001s
I20260709 13:18:56.338143 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:56.336441 17419 rpcz_store.cc:269] 0709 13:18:54.701646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.701781 (+   135us) service_pool.cc:224] Handling call
0709 13:18:56.323136 (+1621355us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b5400573d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:56.323296 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2076267776}
I20260709 13:18:56.344393 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.345332 16744 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:56.317481 19522 raft_consensus.cc:2764] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:18:56.356274 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.356963 16589 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:56.317096 19861 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.357919 16738 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:56.357820 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:56.363016 19678 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.659s	user 0.009s	sys 0.003s
I20260709 13:18:56.347762 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.363487 19678 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.364176 16588 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
W20260709 13:18:56.364670 19916 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.894s	user 0.011s	sys 0.001s
I20260709 13:18:56.365046 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.365648 16588 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:56.363672 19943 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:18:56.367045 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:56.367754 19538 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.353797 16744 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:56.370826 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:56.372011 19935 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.358s	user 0.001s	sys 0.001s
I20260709 13:18:56.374486 19851 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.375485 19904 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.375633 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.376163 19813 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.376356 19902 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.376958 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.376713 19813 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.376691 19470 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.372033 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.365593 19678 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.362394 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.384334 16589 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:56.378676 16744 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:56.381315 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:56.390323 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.347162 19765 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:56.390611 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.394606 16589 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 5.
I20260709 13:18:56.396698 16738 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:56.391716 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.360532 19757 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.404793 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.405843 16589 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.378515 19854 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.417708 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.385183 19807 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.385941 19717 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.399744 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:56.395246 19765 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.421530 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:56.423373 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 296515us
I20260709 13:18:56.426061 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.431918 16589 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:56.432801 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.432777 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.433966 16588 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:56.435775 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.437515 16588 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.418646 19769 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:56.440629 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 270714us
I20260709 13:18:56.442143 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.442708 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.443643 16588 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:56.445617 17419 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:56.424175 19902 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.451656 19902 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.453760 19902 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.441133 19837 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:56.455046 19949 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:18:56.459465 19813 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.460666 19646 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.461414 19646 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.461663 19758 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.440879 19881 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.446219 19769 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.465004 19824 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.468474 19646 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.471010 19805 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.473616 19805 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.474279 16596 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:56.475327 19952 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:18:56.447103 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.424000 19925 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.433751 19765 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:56.481935 19821 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:56.483312 19938 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.465s	user 0.002s	sys 0.000s
I20260709 13:18:56.494587 16667 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:56.503350 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:56.526687 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.484472 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1137) took 1019 ms. Trace:
I20260709 13:18:56.596230 16588 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:56.577814 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.597240 16596 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:56.597435 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:56.597904 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.596467 16736 rpcz_store.cc:276] 0709 13:18:55.465252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.479300 (+ 14048us) service_pool.cc:224] Handling call
0709 13:18:56.484245 (+1004945us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b5400434254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:56.484404 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1797343488}
I20260709 13:18:56.598460 16588 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.598626 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:56.501856 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 59785.1us
I20260709 13:18:56.600286 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.601342 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:56.485137 19905 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:18:56.602999 17419 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:56.602309 16592 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.608189 17419 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:18:56.571839 19758 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.488670 19837 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:56.544313 19830 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:56.608848 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.600275 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.623041 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.623611 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.544054 19862 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.625883 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.487357 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.616598 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.627691 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1145) took 1061 ms. Trace:
I20260709 13:18:56.629065 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.564921 19577 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.597571 19824 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.493508 19560 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.565667 19810 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.626754 16596 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:56.626986 16559 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.562s (SENT)
I20260709 13:18:56.632972 16594 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.641265 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:56.641595 19821 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:56.635212 17409 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:56.643574 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.644667 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.645020 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.645335 16587 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.645634 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.646090 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.646165 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.646447 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.629276 19862 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.626904 19830 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:56.639266 19752 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.628775 16741 rpcz_store.cc:276] 0709 13:18:55.565864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.695295 (+129431us) service_pool.cc:224] Handling call
0709 13:18:55.720270 (+ 24975us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.813047 (+ 92777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.813077 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.838847 (+ 25770us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.934748 (+ 95901us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:55.934787 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.934850 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.476319 (+541469us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:56.627367 (+151048us) raft_consensus.cc:1685] finished
0709 13:18:56.627387 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.627627 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92392,"raft.queue_time_us":92921,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":7854,"spinlock_wait_cycles":74733952,"thread_start_us":52330,"threads_started":1,"wal-append.queue_time_us":168078}
I20260709 13:18:56.642524 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:56.646842 16587 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:56.648754 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:56.659245 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:56.659451 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.660456 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.660609 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.647588 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 530) took 1332 ms. Trace:
I20260709 13:18:56.638585 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:18:56.650480 16594 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:18:56.647323 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.668643 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:56.668926 16634 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.616648 19702 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.676353 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:56.676597 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.677140 17420 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:56.656128 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.666461 17414 rpcz_store.cc:276] 0709 13:18:55.315093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.315245 (+   152us) service_pool.cc:224] Handling call
0709 13:18:56.647419 (+1332174us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:56.647527 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481149824}
I20260709 13:18:56.662603 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.677820 16589 raft_consensus.cc:2368] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9.
W20260709 13:18:56.670750 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43801.6us
I20260709 13:18:56.671294 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.678349 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.678639 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.627099 19837 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:56.677206 16745 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:56.703596 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.658047 16596 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.706086 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.679275 16589 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.707449 16743 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:56.679577 19813 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.754272 19821 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.754388 19813 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:56.754794 19810 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.754995 19813 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.755601 19560 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.679898 19757 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:56.683298 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.683836 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:56.684409 19861 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:56.672394 19803 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:56.685483 19854 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:56.686129 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.680260 19646 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:56.687578 16587 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:56.648048 19824 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:56.810025 19803 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:56.811455 19803 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.669628 19905 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:56.669888 19713 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:56.819324 19713 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:56.819931 19713 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:56.821249 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.822325 17416 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:56.670046 19717 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:18:56.652899 19830 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.675650 19950 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:18:56.647111 19538 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:18:56.827477 19538 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:56.828706 19538 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.647481 19715 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:56.831290 19715 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:56.832417 19715 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.672155 19925 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.670487 19532 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:56.835331 19532 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.835839 19532 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:56.649425 19805 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:18:56.649832 19268 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:18:56.638146 19577 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:56.841676 19577 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.843327 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.844354 19577 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.845104 17410 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:56.655910 19752 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.693783 19702 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.677266 17419 raft_consensus.cc:2410] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:56.847510 19470 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:18:56.847939 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.848147 17419 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:18:56.696442 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.696285 19769 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.648814 19849 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:56.850312 19849 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:56.850483 19769 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.652220 19862 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:56.851046 19143 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.851711 19752 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.852108 19851 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:56.683223 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 1989) took 1222 ms. Trace:
I20260709 13:18:56.852206 19752 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.852694 19851 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.853346 19837 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.854272 19752 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:56.720695 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:56.724643 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:56.855286 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 130764us
I20260709 13:18:56.804291 16596 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 4. Current term is 5. Ops: []
I20260709 13:18:56.704495 16741 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:18:56.807426 17409 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:56.671753 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:56.762594 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:56.729231 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:56.791535 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 113247us
I20260709 13:18:56.857954 17420 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:56.803084 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.759667 17415 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:56.858776 17412 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:18:56.792930 19549 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.000s	sys 0.033s
I20260709 13:18:56.854628 19851 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:56.851100 19860 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.851859 19143 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.718914 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:56.855700 19268 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:56.754885 19821 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.795625 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.858961 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:56.852702 16590 rpcz_store.cc:276] 0709 13:18:55.460839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.460952 (+   113us) service_pool.cc:224] Handling call
0709 13:18:56.683026 (+1222074us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400027e54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:56.683162 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157907840}
I20260709 13:18:56.801002 19845 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:56.797881 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:56.803035 19646 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:56.874848 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 112371us
I20260709 13:18:56.763515 19861 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.804464 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.898170 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:56.899255 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:18:56.883466 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 211760us
I20260709 13:18:56.900104 17412 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:18:56.809090 19932 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.790s	user 0.001s	sys 0.003s
W20260709 13:18:56.871261 19549 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.000s	sys 0.033s
W20260709 13:18:56.882504 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86949.6us
I20260709 13:18:56.886029 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:56.901491 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:56.904474 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1314) took 1895 ms. Trace:
I20260709 13:18:56.909368 17376 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.909328 16668 rpcz_store.cc:276] 0709 13:18:55.008735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.515030 (+506295us) service_pool.cc:224] Handling call
0709 13:18:55.515388 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.634489 (+119101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.634519 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.655171 (+ 20652us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.843247 (+188076us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:55.843295 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.892789 (+1049494us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54003df4d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:56.892862 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.892925 (+    63us) raft_consensus.cc:1685] finished
0709 13:18:56.892968 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.904378 (+ 11410us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":877,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9338,"spinlock_wait_cycles":273102720,"thread_start_us":138300,"threads_started":1,"wal-append.queue_time_us":228558}
I20260709 13:18:56.874661 19560 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.875025 19804 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:56.907459 16590 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:56.911355 17416 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:56.882426 19822 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:18:56.758812 19765 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:56.927821 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.615s (SENT) [suppressed 12 similar messages]
I20260709 13:18:56.928004 16589 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:56.897542 19646 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:56.939954 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:56.893365 19862 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:56.893030 19821 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.882958 19769 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:18:56.946162 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 424) took 1915 ms. Trace:
I20260709 13:18:56.981148 16739 rpcz_store.cc:276] 0709 13:18:55.030594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.036961 (+  6367us) service_pool.cc:224] Handling call
0709 13:18:55.037313 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:55.985441 (+948128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.945871 (+960430us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54003d0254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:56.946081 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976857472}
I20260709 13:18:56.893260 19845 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.986084 19845 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:56.957445 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55664.6us
I20260709 13:18:56.956612 19822 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.893584 19805 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:56.903092 19861 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:18:56.956550 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62291.7us
I20260709 13:18:56.958654 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1146) took 1392 ms. Trace:
I20260709 13:18:56.903117 19925 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:56.942045 19757 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:18:56.942844 17413 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:56.884773 19860 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:56.989018 16588 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:56.938048 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 584) took 2143 ms. Trace:
I20260709 13:18:56.956223 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:56.988305 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:56.992864 19822 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:56.977349 19862 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:56.958841 19968 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:56.971213 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:57.048713 16589 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:56.997713 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:56.976320 19854 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:18:57.050117 16593 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:56.998989 16742 rpcz_store.cc:276] 0709 13:18:55.566163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.697461 (+131298us) service_pool.cc:224] Handling call
0709 13:18:55.718754 (+ 21293us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:55.987036 (+268282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.987085 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:55.987813 (+   728us) log.cc:844] Serialized 52 byte log entry
0709 13:18:56.177962 (+190149us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:56.178019 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.936391 (+758372us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400306954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:56.936446 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.936518 (+    72us) raft_consensus.cc:1685] finished
0709 13:18:56.936532 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:56.958568 (+ 22036us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2281,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4750,"spinlock_wait_cycles":411298688,"thread_start_us":154718,"threads_started":1,"wal-append.queue_time_us":201685}
I20260709 13:18:57.003264 19925 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:57.011353 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2626) took 1375 ms. Trace:
I20260709 13:18:56.955969 19577 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:18:57.008698 17411 rpcz_store.cc:276] 0709 13:18:54.795003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:54.804900 (+  9897us) service_pool.cc:224] Handling call
0709 13:18:56.323270 (+1518370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:56.335803 (+ 12533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:56.937847 (+602044us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400573d50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:56.937983 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3491137920}
I20260709 13:18:56.943435 19560 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:56.956486 19804 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.018939 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:57.019821 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:57.020604 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:57.021380 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:57.036422 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.055296 16595 rpcz_store.cc:276] 0709 13:18:55.635482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:55.638571 (+  3089us) service_pool.cc:224] Handling call
0709 13:18:55.699042 (+ 60471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:55.700021 (+   979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:55.700052 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:55.703148 (+  3096us) log.cc:844] Serialized 26 byte log entry
0709 13:18:55.931959 (+228811us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:55.932000 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:55.932062 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:56.808336 (+876274us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:57.007365 (+199029us) raft_consensus.cc:1685] finished
0709 13:18:57.007394 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.011276 (+  3882us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26430,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":58188032,"thread_start_us":228343,"threads_started":1,"wal-append.queue_time_us":289630}
I20260709 13:18:57.062569 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:56.943365 19849 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:57.075898 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 87668.2us
I20260709 13:18:57.077461 19939 raft_consensus.cc:2764] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:57.078737 19854 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:18:57.076880 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:57.079826 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:57.080992 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:57.081022 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:57.083920 16671 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:57.079149 19854 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:57.104512 19905 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:57.105265 19939 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:18:57.105419 19854 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.047619 19968 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.106529 19968 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.109403 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.111542 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.112118 16742 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:18:57.049523 19862 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.018067 17409 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:57.059137 19925 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:57.116138 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.129325 16741 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.115370 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:57.130144 16739 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:18:57.130594 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:57.125743 16662 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:57.127043 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:57.130630 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 244661us
I20260709 13:18:57.132217 16745 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:18:57.133822 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:57.135041 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:57.135792 16663 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:57.114913 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:57.162253 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.106768 19854 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:57.170078 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 171843us
W20260709 13:18:57.169014 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 241209us
I20260709 13:18:57.180958 16745 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
W20260709 13:18:57.186019 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.186969 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:18:57.123489 19851 consensus_queue.cc:1175] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "efa00eb5e7404570958fdebb6cd077e8" responder_term: 6 status { last_received { term: 6 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 4 } }
I20260709 13:18:57.211704 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:57.214311 19470 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.208209 17410 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:18:57.127795 19905 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:57.205672 16671 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
W20260709 13:18:57.170287 19927 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:18:57.216570 16742 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:57.219776 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:57.163511 19522 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:57.164402 19973 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:57.227733 17410 pending_rounds.cc:85] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 6
I20260709 13:18:57.229261 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:57.245357 17410 pending_rounds.cc:107] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 8.7
I20260709 13:18:57.245831 17410 raft_consensus.cc:2917] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Aborting config change with OpId 8.7: Aborted: Op aborted by new leader
I20260709 13:18:57.246452 17410 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:18:57.231582 16593 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:57.237380 16596 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:57.262841 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:57.155841 19968 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:57.238150 16663 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:57.241079 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:57.257658 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:57.266559 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:57.267434 16671 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:57.235268 19769 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:57.224802 19940 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:57.228932 19143 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:57.217799 19975 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:57.273458 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:18:57.286257 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:57.269590 19911 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.313426 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:18:57.339430 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.234663 19470 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.344985 19143 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:18:57.281960 19902 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:57.248415 19522 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.342084 16708 leader_election.cc:400] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9.
I20260709 13:18:57.351025 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:57.353523 16708 leader_election.cc:403] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:57.354707 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:57.267100 19758 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:18:57.318327 19891 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:57.376236 16671 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:57.281860 19975 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:18:57.367503 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.265272 19982 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:18:57.265602 19973 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:57.346380 19642 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:57.274998 19769 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:57.394104 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.352464 19902 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:57.374236 19538 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:18:57.370767 19904 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9.
I20260709 13:18:57.371156 19143 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:57.375751 19891 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:18:57.422621 19268 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.037s	sys 0.036s
I20260709 13:18:57.382725 19860 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:57.440191 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 307300us
I20260709 13:18:57.389112 19975 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:57.425072 19804 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:57.427209 19642 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.431744 19769 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.310617 19940 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:57.434386 19902 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:18:57.466352 17376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.019s	sys 0.031s
I20260709 13:18:57.437922 19538 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:57.359051 19905 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:57.477953 17376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.020s	sys 0.032s
I20260709 13:18:57.447881 19891 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.486548 17412 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:57.456238 19983 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:57.446262 19268 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:57.466650 19642 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:57.496528 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 381692us
I20260709 13:18:57.499032 16741 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
W20260709 13:18:57.500257 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 254298us
I20260709 13:18:57.501433 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:57.510277 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:57.488730 19538 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:57.466917 19904 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:18:57.488073 19765 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:57.507007 19577 raft_consensus.cc:670] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: failed to trigger leader election: Illegal state: only voting members can start elections: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } [suppressed 4 similar messages]
W20260709 13:18:57.527181 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 260618us
I20260709 13:18:57.495692 19983 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:57.535435 19983 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:57.536526 19522 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:57.567191 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:57.523758 19972 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:18:57.643178 17412 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:57.644520 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:57.645747 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:57.647042 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:57.643656 17412 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:57.666038 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2650) took 1046 ms. Trace:
I20260709 13:18:57.655719 19758 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:57.704011 16595 rpcz_store.cc:276] 0709 13:18:56.619677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.123709 (+504032us) service_pool.cc:224] Handling call
0709 13:18:57.124018 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:57.124224 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:18:57.429171 (+304947us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.664437 (+235266us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:57.664522 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:18:57.664783 (+   261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.665150 (+   367us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.665774 (+   624us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":101443,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":532362,"threads_started":2}
W20260709 13:18:57.670492 16743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.004s	sys 0.000s
I20260709 13:18:57.537196 19522 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:57.685998 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 151056us
I20260709 13:18:57.534514 19926 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:57.724820 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:57.726140 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:57.727169 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:57.727783 16593 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:57.537701 19968 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:57.706346 16743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.004s	sys 0.000s
W20260709 13:18:57.708489 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 198792us
W20260709 13:18:57.726316 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.726751 16596 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
W20260709 13:18:57.747848 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.748734 16596 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:18:57.715338 19813 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.960s	user 0.009s	sys 0.004s
I20260709 13:18:57.755541 16596 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:57.707065 19758 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:18:57.703011 19980 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:18:57.729557 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:57.715991 19997 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:57.745103 19907 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:57.764706 19968 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:18:57.777840 19968 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:18:57.778306 19968 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:57.779394 19968 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:57.780872 19968 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:57.751432 20016 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:57.697546 19268 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:57.703768 19577 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:18:57.807659 19577 consensus_queue.cc:1243] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:18:57.807839 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.765856 16595 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:18:57.656170 19143 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.746989 19977 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:57.772126 19985 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 13:18:57.798911 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2580) took 1674 ms. Trace:
I20260709 13:18:57.816442 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.817147 16741 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:57.817492 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:57.819918 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.820785 16742 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:57.770598 19973 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:57.758358 19470 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:57.798808 19532 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.962s	user 0.004s	sys 0.009s
W20260709 13:18:57.769937 19981 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.320s	user 0.000s	sys 0.002s
I20260709 13:18:57.787366 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:57.770246 19997 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:57.822031 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:57.822690 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:57.822943 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.824129 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:57.824648 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:57.827068 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:57.796528 19758 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:57.833664 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:57.679831 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 320759us
I20260709 13:18:57.761826 19813 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:57.790510 19861 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.777s	user 0.009s	sys 0.001s
I20260709 13:18:57.840932 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:57.843194 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 187226us
W20260709 13:18:57.850910 20015 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:18:57.851994 19560 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:57.852535 20012 raft_consensus.cc:2764] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:18:57.859426 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:57.858388 19538 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.859577 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:18:58.329128 16591 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.015s	sys 0.007s
W20260709 13:18:58.441331 16591 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.017s	sys 0.009s
I20260709 13:18:58.735322 16589 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:58.731509 16493 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:18:58.738993 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 282) took 2399 ms. Trace:
W20260709 13:18:58.741036 16588 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 2000) took 2346 ms (client timeout 2361 ms). Trace:
I20260709 13:18:58.739645 16592 rpcz_store.cc:276] 0709 13:18:56.339307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.700150 (+360843us) service_pool.cc:224] Handling call
0709 13:18:56.700536 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:56.761629 (+ 61093us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:56.761665 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:56.803515 (+ 41850us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.255094 (+451579us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:57.313985 (+ 58891us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.319711 (+  5726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.319809 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.438230 (+1118421us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b1800316660. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:58.686536 (+248306us) spinlock_profiling.cc:243] Waited 6.38 ms on lock 0x7b1800316660. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:58.738279 (+ 51743us) raft_consensus.cc:1685] finished
0709 13:18:58.738313 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.738672 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66134,"prepare.queue_time_us":25866,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":3931,"raft.queue_time_us":97401,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":10922,"spinlock_wait_cycles":22516224,"thread_start_us":352728,"threads_started":1,"wal-append.queue_time_us":469020}
W20260709 13:18:58.741482 16588 rpcz_store.cc:269] 0709 13:18:56.393952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.623380 (+229428us) service_pool.cc:224] Handling call
0709 13:18:58.740549 (+2117169us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540008acd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:58.740802 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838610688}
W20260709 13:18:58.441812 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 794805us
I20260709 13:18:58.746090 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2649) took 2169 ms. Trace:
I20260709 13:18:58.746250 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:57.985711 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.481396 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:58.753211 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.753070 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 271584us
W20260709 13:18:58.256520 16636 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.007s	sys 0.014s
I20260709 13:18:58.754791 16634 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:58.755365 16636 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.027s	user 0.009s	sys 0.015s
W20260709 13:18:58.756110 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 1.02832e+06us
I20260709 13:18:58.446096 16669 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:58.758499 16669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1342) took 1240 ms (client timeout 1652 ms). Trace:
W20260709 13:18:58.759042 16669 rpcz_store.cc:269] 0709 13:18:57.517643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.787099 (+269456us) service_pool.cc:224] Handling call
0709 13:18:58.758286 (+971187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:58.761214 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.761106 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:58.746701 16589 rpcz_store.cc:276] 0709 13:18:56.577015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.049520 (+472505us) service_pool.cc:224] Handling call
0709 13:18:57.049761 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.736084 (+1686323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.745627 (+  9543us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400052654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:58.745975 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3874965760}
I20260709 13:18:58.738266 16493 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 5 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:18:58.751072 16662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1326) took 1730 ms (client timeout 1986 ms). Trace:
W20260709 13:18:58.762542 16662 rpcz_store.cc:269] 0709 13:18:57.020132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.021141 (+  1009us) service_pool.cc:224] Handling call
0709 13:18:58.750359 (+1729218us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b540009d654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:58.750690 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143668096}
W20260709 13:18:58.754853 16708 connection.cc:256] RPC call timeout handler was delayed by -1.3163329830001658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 13:18:58.763971 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:58.777210 16711 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.027s	sys 0.031s
I20260709 13:18:58.777771 16662 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 13:18:58.777923 16711 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.027s	sys 0.031s
I20260709 13:18:58.778673 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1956) took 1644 ms. Trace:
I20260709 13:18:58.779242 16662 rpcz_store.cc:276] 0709 13:18:57.134322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.763523 (+1629201us) service_pool.cc:224] Handling call
0709 13:18:58.778528 (+ 15005us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:58.780578 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.780777 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:58.781948 16662 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 4.
I20260709 13:18:58.781698 16493 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 6, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 6 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:18:58.756399 16671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1333) took 1732 ms (client timeout 2086 ms). Trace:
W20260709 13:18:58.780771 16494 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 234) took 11466 ms (client timeout 15000 ms). Trace:
W20260709 13:18:58.783214 16671 rpcz_store.cc:269] 0709 13:18:57.023906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.354417 (+330511us) service_pool.cc:224] Handling call
0709 13:18:58.755960 (+1401543us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b540001b154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:58.756206 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47801216}
I20260709 13:18:58.785544 16493 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:58.782634 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1957) took 1610 ms. Trace:
W20260709 13:18:58.790534 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44842.7us
I20260709 13:18:58.791064 16662 rpcz_store.cc:276] 0709 13:18:57.172265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.780270 (+1608005us) service_pool.cc:224] Handling call
0709 13:18:58.782547 (+  2277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.792013 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:18:58.789906 16493 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 6, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 6 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:58.754451 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:58.795989 16663 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1955) took 1706 ms (client timeout 2138 ms). Trace:
I20260709 13:18:58.795504 16493 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:18:58.796562 16663 rpcz_store.cc:269] 0709 13:18:57.089254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.753551 (+1664297us) service_pool.cc:224] Handling call
0709 13:18:58.795890 (+ 42339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.449084 16736 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:58.800140 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:57.859611 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:58.445135 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:58.802104 16739 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:57.871943 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:58.553675 17409 consensus_meta.cc:287] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.386s	user 0.010s	sys 0.002s
I20260709 13:18:58.803514 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:58.804440 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:58.804808 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1167) took 1684 ms. Trace:
I20260709 13:18:58.804970 16739 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:58.447855 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:58.446725 16744 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:58.802768 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:57.814275 16740 rpcz_store.cc:276] 0709 13:18:56.124238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.127390 (+  3152us) service_pool.cc:224] Handling call
0709 13:18:56.127701 (+   311us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:18:56.184405 (+ 56704us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:18:56.682424 (+498019us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b18003a92a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf4455 00007f32835dc3c7 00007f327fbee9a7 00007f327fbfe0bf 00007f327fbfe038
0709 13:18:56.906320 (+223896us) log.cc:844] Serialized 14 byte log entry
0709 13:18:57.685083 (+778763us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:57.687175 (+  2092us) log.cc:844] Serialized 52 byte log entry
0709 13:18:57.687317 (+   142us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:57.687356 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.687419 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:57.798362 (+110943us) raft_consensus.cc:1685] finished
0709 13:18:57.798515 (+   153us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:57.798827 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":165918,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":392847,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":3848,"spinlock_wait_cycles":6953344,"thread_start_us":948705,"threads_started":3,"wal-append.queue_time_us":825753}
W20260709 13:18:58.499660 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 658765us
W20260709 13:18:58.441512 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:58.076124 16737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.000s	sys 0.035s
I20260709 13:18:58.806836 16745 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:18:58.839651 16737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.597s	user 0.002s	sys 0.052s
I20260709 13:18:58.839946 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2600) took 1016 ms. Trace:
I20260709 13:18:58.840408 16745 rpcz_store.cc:276] 0709 13:18:57.823759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.805643 (+981884us) service_pool.cc:224] Handling call
0709 13:18:58.806276 (+   633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:58.839622 (+ 33346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.839864 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:57.873840 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:58.515515 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:58.841784 20007 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.865247 17415 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:58.866004 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 548) took 1025 ms. Trace:
I20260709 13:18:58.865722 20007 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.869001 20007 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:58.440740 17416 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:58.443034 17419 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:18:58.532207 19268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.004s	sys 0.013s
I20260709 13:18:58.259918 19522 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:58.886327 19522 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.480744 19532 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.647624 19538 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:58.628885 19549 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:18:58.889220 19549 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:58.507906 19560 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.891469 19560 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:57.870010 19646 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.889s	user 0.008s	sys 0.000s
I20260709 13:18:58.893615 19646 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.651315 19642 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.136s	user 0.011s	sys 0.001s
I20260709 13:18:58.896694 19642 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.262933 19713 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.442s	user 0.014s	sys 0.000s
W20260709 13:18:58.551988 19769 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.033s	user 0.008s	sys 0.029s
I20260709 13:18:58.900884 19713 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.645617 19813 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:57.858637 19821 consensus_meta.cc:287] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.871s	user 0.002s	sys 0.006s
W20260709 13:18:58.511452 19822 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.458s	user 0.009s	sys 0.018s
I20260709 13:18:58.911134 16736 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:18:58.912354 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:58.912849 19538 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.913563 19538 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.397956 19854 consensus_meta.cc:287] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.222s	user 0.009s	sys 0.000s
I20260709 13:18:58.908107 19837 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:18:58.916606 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:58.500033 19861 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.920975 19765 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.921923 19765 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.135697 19860 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.003s	sys 0.009s
I20260709 13:18:58.923121 19268 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.923578 19860 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.394s	user 0.008s	sys 0.023s
W20260709 13:18:58.624305 19862 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.460s	user 0.011s	sys 0.000s
I20260709 13:18:58.925410 19862 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:58.685303 19891 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.179s	user 0.011s	sys 0.000s
I20260709 13:18:58.928489 19891 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:58.677030 19902 consensus_meta.cc:287] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.187s	user 0.006s	sys 0.005s
I20260709 13:18:58.890673 19532 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:58.931610 19902 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.932360 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:58.931604 19713 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:57.862644 19904 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:58.911207 19821 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.944382 19904 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:58.945456 16670 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:58.946391 19717 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:18:58.421119 19911 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.075s	user 0.011s	sys 0.000s
I20260709 13:18:58.947638 19911 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:18:58.262192 19925 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.094s	user 0.008s	sys 0.004s
I20260709 13:18:58.948628 16738 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:18:58.948719 19821 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:58.931533 19646 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:58.949030 19925 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.950349 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2603) took 1063 ms. Trace:
W20260709 13:18:58.329375 19926 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.004s	sys 0.017s
I20260709 13:18:58.443753 19939 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:58.951323 19926 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.008s	sys 0.022s
I20260709 13:18:58.952019 19926 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:18:58.952884 20022 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.951195 19940 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:58.690546 19969 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 1.579s	user 0.003s	sys 0.034s
I20260709 13:18:58.953720 19926 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.183276 19975 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.002s
I20260709 13:18:58.959842 19975 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:57.839856 19977 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:57.840471 19980 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:18:58.484997 19983 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.949s	user 0.011s	sys 0.001s
W20260709 13:18:58.630569 19993 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:18:58.479686 19997 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:58.980682 19538 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:58.983615 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1330) took 1961 ms. Trace:
I20260709 13:18:58.984215 16589 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:18:57.866990 20000 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:18:58.136463 20005 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.444s
I20260709 13:18:58.391721 20004 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.977308 19983 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.987175 20004 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.989600 20004 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:58.624934 20009 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.991088 20009 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.992167 19891 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:58.984067 16664 rpcz_store.cc:276] 0709 13:18:57.021585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.021711 (+   126us) service_pool.cc:224] Handling call
0709 13:18:57.022074 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.022463 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.022502 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.047694 (+ 25192us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.437683 (+389989us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:18:57.437730 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.437797 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.983094 (+1545297us) raft_consensus.cc:1685] finished
0709 13:18:58.983142 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.983377 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":191597,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":310837,"threads_started":1,"wal-append.queue_time_us":1318936}
I20260709 13:18:58.993695 19902 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:58.805264 16743 rpcz_store.cc:276] 0709 13:18:57.120545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.120726 (+   181us) service_pool.cc:224] Handling call
0709 13:18:57.121203 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.121678 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.121712 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.127754 (+  6042us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.739982 (+612228us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:57.740023 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.740114 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.804209 (+1064095us) raft_consensus.cc:1685] finished
0709 13:18:58.804264 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.804523 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2032,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":1827,"thread_start_us":611868,"threads_started":1,"wal-append.queue_time_us":575218}
W20260709 13:18:58.783435 16494 rpcz_store.cc:269] 0709 13:18:47.314159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.958035 (+643876us) service_pool.cc:224] Handling call
0709 13:18:58.780421 (+10822386us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:50.738274 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:50.828549 (+ 90275us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:50.828615 (+    66us) write_op.cc:435] Acquired schema lock
0709 13:18:50.828645 (+    30us) tablet.cc:662] Decoding operations
0709 13:18:51.049020 (+220375us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:51.066497 (+ 17477us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:51.067993 (+  1496us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:18:51.254445 (+186452us) tablet.cc:701] Row locks acquired
0709 13:18:51.275234 (+ 20789us) write_op.cc:260] PREPARE: finished
0709 13:18:51.318111 (+ 42877us) write_op.cc:270] Start()
0709 13:18:51.343020 (+ 24909us) write_op.cc:276] Timestamp: P: 1783603131306159 usec, L: 0
0709 13:18:51.355125 (+ 12105us) op_driver.cc:348] REPLICATION: starting
0709 13:18:51.540085 (+184960us) log.cc:844] Serialized 9551 byte log entry
0709 13:18:52.038336 (+498251us) op_driver.cc:464] REPLICATION: finished
0709 13:18:52.652581 (+614245us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b18002a2c80. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:18:52.652768 (+   187us) write_op.cc:301] APPLY: starting
0709 13:18:52.652868 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:18:52.653274 (+   406us) tablet.cc:1370] finished BulkCheckPresence
0709 13:18:52.653314 (+    40us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:18:53.020060 (+366746us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:18:53.020540 (+   480us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 13:18:53.020600 (+    60us) write_op.cc:312] APPLY: finished
0709 13:18:53.021101 (+   501us) log.cc:844] Serialized 158 byte log entry
0709 13:18:53.457796 (+436695us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:53.498673 (+ 40877us) write_op.cc:454] Released schema lock
0709 13:18:53.556140 (+ 57467us) write_op.cc:341] FINISH: Updating metrics
0709 13:18:53.606273 (+ 50133us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:18:53.612737 (+  6464us) write_op.cc:454] Released schema lock
0709 13:18:54.284017 (+671280us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
Metrics: {"mutex_wait_us":1107064,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":107122,"apply.run_cpu_time_us":28779,"apply.run_wall_time_us":934101,"num_ops":23,"prepare.queue_time_us":125737,"prepare.run_cpu_time_us":37048,"prepare.run_wall_time_us":996949,"raft.queue_time_us":136447,"raft.run_cpu_time_us":24570,"raft.run_wall_time_us":117494,"replication_time_us":682978,"spinlock_wait_cycles":9219072,"thread_start_us":837985,"threads_started":5,"wal-append.queue_time_us":838364,"wal-append.run_cpu_time_us":44273,"wal-append.run_wall_time_us":3093200}]]}
W20260709 13:18:58.816972 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59256.2us
I20260709 13:18:58.807448 16741 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:58.820994 16742 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:18:58.804457 17409 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:58.821329 19470 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:59.010396 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.013607 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1321) took 1995 ms. Trace:
I20260709 13:18:59.014286 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2604) took 1135 ms. Trace:
I20260709 13:18:58.805258 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:18:58.832496 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 604) took 1366 ms. Trace:
I20260709 13:18:59.014684 16742 rpcz_store.cc:276] 0709 13:18:57.872213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.802228 (+930015us) service_pool.cc:224] Handling call
0709 13:18:59.007992 (+205764us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.833191 19851 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:58.826637 19804 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:58.835175 20024 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:18:58.834038 20021 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.834717 20026 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:18:58.838339 19582 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.838654 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2641) took 2644 ms. Trace:
I20260709 13:18:58.849527 19881 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.856426 19963 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:58.858049 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:58.858246 20043 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.826946 19999 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:58.859360 20037 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:58.862123 20018 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:58.862828 19905 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.863763 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 549) took 1023 ms. Trace:
I20260709 13:18:58.876180 20035 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:58.876731 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:58.866489 17415 rpcz_store.cc:276] 0709 13:18:57.840110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.515173 (+675063us) service_pool.cc:224] Handling call
0709 13:18:58.865881 (+350708us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:58.852278 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:58.878199 20038 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.878371 16493 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 5, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 5 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:58.880060 20036 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.901048 19769 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.131817 19805 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:59.046267 19963 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.049099 19805 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.049322 20038 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.050897 19963 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.051891 20038 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.052357 19805 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.053206 16740 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:59.052681 19999 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:58.902815 16666 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:59.054723 20018 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.905646 19998 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 1.129s	user 0.000s	sys 0.005s
I20260709 13:18:59.021252 16591 rpcz_store.cc:276] 0709 13:18:56.194318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.195159 (+   841us) service_pool.cc:224] Handling call
0709 13:18:56.195676 (+   517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:56.196157 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:56.935951 (+739794us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b18003770e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf4455 00007f32835dc3c7 00007f327fbee9a7 00007f327fbfe0bf 00007f327fbfe038
0709 13:18:57.268198 (+332247us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.735923 (+1467725us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:58.737474 (+  1551us) log.cc:844] Serialized 52 byte log entry
0709 13:18:58.737795 (+   321us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:58.737861 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.737983 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.838082 (+100099us) raft_consensus.cc:1685] finished
0709 13:18:58.838218 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.838455 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27038,"prepare.queue_time_us":86491,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":94048,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":25525,"spinlock_wait_cycles":8312448,"thread_start_us":1722861,"threads_started":2,"wal-append.queue_time_us":1313279}
I20260709 13:18:58.906100 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1168) took 1783 ms. Trace:
I20260709 13:18:58.911834 19822 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:58.434392 19845 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.443s	user 0.011s	sys 0.000s
I20260709 13:18:58.924153 19268 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.067524 19845 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:58.926671 19642 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:59.069185 16667 raft_consensus.cc:2368] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9.
I20260709 13:18:59.069185 19268 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.069895 16671 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:59.070204 19845 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.071018 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.071877 19925 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.072293 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.072710 19905 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.073727 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:58.934841 16739 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:18:59.075196 20045 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:58.953874 16669 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:58.509379 19973 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:59.078006 19996 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:18:58.686033 19968 consensus_meta.cc:287] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.002s
I20260709 13:18:59.080406 16587 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:58.953749 20022 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.952101 19939 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:58.955255 20025 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:18:58.950830 16744 rpcz_store.cc:276] 0709 13:18:57.871958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.807875 (+935917us) service_pool.cc:224] Handling call
0709 13:18:58.808452 (+   577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:58.817805 (+  9353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:58.817847 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:18:58.821638 (+  3791us) log.cc:844] Serialized 52 byte log entry
0709 13:18:58.846809 (+ 25171us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:18:58.846847 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.846968 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.908625 (+ 61657us) raft_consensus.cc:1685] finished
0709 13:18:58.908644 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.935499 (+ 26855us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55455,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":271,"thread_start_us":33186,"threads_started":2,"wal-append.queue_time_us":78335}
I20260709 13:18:58.957823 16592 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:58.946065 19904 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.087437 19939 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.087500 19904 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.056530 19975 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.089998 20022 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.010409 17414 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:18:59.011480 20044 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.012089 19470 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.091352 20044 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:18:59.095283 19470 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.097162 20044 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:59.013461 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.013113 17412 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.099491 17412 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:58.835249 20027 raft_consensus.cc:2764] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:18:58.834373 20019 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:59.013967 16665 rpcz_store.cc:276] 0709 13:18:57.018445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.018599 (+   154us) service_pool.cc:224] Handling call
0709 13:18:59.013295 (+1994696us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b5400347454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.013459 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4169912064}
I20260709 13:18:58.989610 19983 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:58.994326 20009 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.019594 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2606) took 1144 ms. Trace:
W20260709 13:18:59.028746 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 110158us
I20260709 13:18:59.017308 19804 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.103578 20028 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.105000 19804 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:59.019905 20026 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.076115 19862 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.030704 17411 rpcz_store.cc:276] 0709 13:18:57.840355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.842165 (+  1810us) service_pool.cc:224] Handling call
0709 13:18:58.863530 (+1021365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.015059 17410 rpcz_store.cc:276] 0709 13:18:57.465820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.465984 (+   164us) service_pool.cc:224] Handling call
0709 13:18:57.490671 (+ 24687us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.491036 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.491062 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.491495 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.710677 (+219182us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:18:57.710903 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.711054 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.831996 (+1120942us) raft_consensus.cc:1685] finished
0709 13:18:58.832030 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.832278 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":168468,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":13774,"thread_start_us":172721,"threads_started":1,"wal-append.queue_time_us":1114666}
I20260709 13:18:59.038782 20036 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.039268 19854 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.039696 17419 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:18:59.043656 16663 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.119222 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.044981 16493 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:18:59.047156 19758 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.047647 20035 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.048076 19769 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.055160 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:58.903894 19560 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.059816 16737 rpcz_store.cc:276] 0709 13:18:57.122892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.123051 (+   159us) service_pool.cc:224] Handling call
0709 13:18:57.164864 (+ 41813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.165224 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.165256 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.168067 (+  2811us) log.cc:844] Serialized 26 byte log entry
0709 13:18:58.840246 (+1672179us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:18:58.855770 (+ 15524us) log.cc:844] Serialized 14 byte log entry
0709 13:18:58.898318 (+ 42548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:58.905726 (+  7408us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.905799 (+    73us) raft_consensus.cc:1685] finished
0709 13:18:58.905816 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:58.906027 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54746,"prepare.queue_time_us":100089,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":35628,"raft.queue_time_us":41265,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":8581,"thread_start_us":1612012,"threads_started":2,"wal-append.queue_time_us":1374555}
I20260709 13:18:59.063513 19881 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:59.135537 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 80647.1us
I20260709 13:18:59.135401 19854 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.136555 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.137153 19881 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.137794 19881 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:18:59.138051 19854 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.138422 19769 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:18:59.043506 20021 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.066556 16594 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:59.139001 19769 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:59.031437 19582 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.140203 19769 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.141533 19582 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.075060 20043 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.141952 16588 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:59.079332 16738 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:18:59.143189 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.138073 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.143833 16588 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:18:59.144094 20043 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.079989 19996 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:18:59.082741 19822 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:59.037357 17409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 542) took 2137 ms (client timeout 1642 ms). Trace:
W20260709 13:18:59.145288 17409 rpcz_store.cc:269] 0709 13:18:56.894934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.897600 (+  2666us) service_pool.cc:224] Handling call
0709 13:18:59.028743 (+2131143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.077739 20045 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.035534 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.103948 16741 rpcz_store.cc:276] 0709 13:18:57.872675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.447545 (+574870us) service_pool.cc:224] Handling call
0709 13:18:59.014231 (+566686us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.110878 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.109145 20026 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:18:59.115195 16559 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:59.116359 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.101150 20019 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.150460 16592 raft_consensus.cc:2410] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:18:59.117043 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.151225 16708 leader_election.cc:400] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:18:59.151647 16708 leader_election.cc:403] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:18:59.151971 20019 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.119190 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2605) took 1246 ms. Trace:
I20260709 13:18:59.152944 20017 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:59.153357 19881 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:18:59.154151 20017 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:59.154336 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.119699 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:59.155110 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.155277 16595 raft_consensus.cc:2376] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9. Re-sending same reply.
I20260709 13:18:59.155380 20009 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:59.120693 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.155920 20009 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:59.156381 16587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2652) took 2451 ms (client timeout 2152 ms). Trace:
I20260709 13:18:59.120635 19549 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:18:59.156840 16587 rpcz_store.cc:269] 0709 13:18:56.705261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.804122 (+ 98861us) service_pool.cc:224] Handling call
0709 13:18:59.156115 (+2351993us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b5400157a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.156291 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5008404736}
I20260709 13:18:59.157717 19549 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.157922 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:59.121841 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 264236us
I20260709 13:18:59.122032 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.122773 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.124421 19861 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.165000 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:18:59.165768 16669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1961) took 1935 ms (client timeout 2081 ms). Trace:
I20260709 13:18:59.165947 20044 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:59.166210 16669 rpcz_store.cc:269] 0709 13:18:57.230337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.760668 (+1530331us) service_pool.cc:224] Handling call
0709 13:18:59.165534 (+404866us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540000ffd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.165688 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":397356928}
I20260709 13:18:59.165941 20018 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:59.167521 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.168213 16669 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:59.168398 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.168632 16664 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:59.169281 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.127683 19849 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:59.169664 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.131981 17419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 547) took 1299 ms. Trace:
I20260709 13:18:59.132426 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1337) took 2080 ms. Trace:
I20260709 13:18:59.171007 16663 rpcz_store.cc:276] 0709 13:18:57.048491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.823157 (+1774666us) service_pool.cc:224] Handling call
0709 13:18:59.128337 (+305180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.053434 19977 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.170899 17419 rpcz_store.cc:276] 0709 13:18:57.835678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.859164 (+ 23486us) service_pool.cc:224] Handling call
0709 13:18:59.131873 (+1272709us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.140645 20021 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.079986 20037 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.092944 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:18:59.146083 17411 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:18:59.154628 20017 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:59.124553 19997 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.151541 16589 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:18:59.118355 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1325) took 2098 ms. Trace:
I20260709 13:18:59.156510 20009 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:59.105237 20028 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.152586 16736 rpcz_store.cc:276] 0709 13:18:57.872443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.799465 (+927022us) service_pool.cc:224] Handling call
0709 13:18:59.119024 (+319559us) spinlock_profiling.cc:243] Waited 97.1 ms on lock 0x7b5400426154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.119136 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223264128}
I20260709 13:18:59.160887 19822 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:59.161303 16593 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.162026 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.169323 20018 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:18:59.126463 17416 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:59.134547 20035 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.110036 19905 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.184224 19861 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:59.185670 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 155966us
I20260709 13:18:59.186301 19560 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:18:59.211874 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2598) took 1383 ms. Trace:
I20260709 13:18:59.191226 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.145558 19996 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.205746 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.219280 16662 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:18:59.197813 16670 rpcz_store.cc:276] 0709 13:18:57.019820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.020310 (+   490us) service_pool.cc:224] Handling call
0709 13:18:59.118102 (+2097792us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54001f6654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.118271 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4328601856}
I20260709 13:18:59.216671 16738 rpcz_store.cc:276] 0709 13:18:57.823178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.824491 (+  1313us) service_pool.cc:224] Handling call
0709 13:18:59.172721 (+1348230us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b54003edfd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.206422 (+ 33701us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168177024}
I20260709 13:18:59.220494 16665 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.220546 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 545) took 1388 ms. Trace:
I20260709 13:18:59.221231 17416 rpcz_store.cc:276] 0709 13:18:57.832052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.832168 (+   116us) service_pool.cc:224] Handling call
0709 13:18:59.220467 (+1388299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.222947 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1960) took 2001 ms. Trace:
I20260709 13:18:59.223308 16662 rpcz_store.cc:276] 0709 13:18:57.218658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.174092 (+1955434us) service_pool.cc:224] Handling call
0709 13:18:59.219669 (+ 45577us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.209161 19822 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:59.233317 17412 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:18:59.216465 19560 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:18:59.222836 19830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.970s	user 0.009s	sys 0.022s
I20260709 13:18:59.226655 19822 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.261754 19560 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:18:59.262313 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 68111.1us
I20260709 13:18:59.268334 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 606) took 1752 ms. Trace:
I20260709 13:18:59.268841 17412 rpcz_store.cc:276] 0709 13:18:57.515508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.515655 (+   147us) service_pool.cc:224] Handling call
0709 13:18:57.643047 (+127392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:58.994003 (+1350956us) spinlock_profiling.cc:243] Waited 64.8 ms on lock 0x7b5400549fa0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb6206f 00007f327fbdaf60 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62
0709 13:18:59.233864 (+239861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.268277 (+ 34413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":148952320}
I20260709 13:18:59.269609 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.278728 16740 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:59.279346 16736 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:59.282562 20044 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:18:59.285010 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1329) took 2263 ms. Trace:
W20260709 13:18:59.287498 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 58343.1us
W20260709 13:18:59.264605 19830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.155s	user 0.010s	sys 0.022s
I20260709 13:18:59.289165 16666 rpcz_store.cc:276] 0709 13:18:57.021242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.021369 (+   127us) service_pool.cc:224] Handling call
0709 13:18:57.021615 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.054214 (+2032599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.284672 (+230458us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400145854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:59.284904 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4325287680}
I20260709 13:18:59.298493 20020 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:18:59.278743 19577 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:18:59.264199 19861 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.311472 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:59.316193 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.316340 20019 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:18:59.317652 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.310604 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:18:59.334882 20051 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:18:59.345705 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:59.347025 19582 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:59.357811 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.362576 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.363566 16665 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:18:59.365090 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52850.6us
I20260709 13:18:59.365399 16590 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:18:59.366210 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:18:59.367434 20048 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 13:18:59.364521 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.371675 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.372910 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.383955 20037 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:18:59.387261 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49295.7us
I20260709 13:18:59.391106 19977 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:59.391825 20003 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:18:59.397857 20003 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:18:59.398236 20003 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:59.328480 20036 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.392709 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.432005 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.401219 19860 raft_consensus.cc:2764] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:18:59.437716 16593 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:18:59.436187 16667 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1323) took 2417 ms (client timeout 1815 ms). Trace:
I20260709 13:18:59.446452 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.446928 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:59.445382 16667 rpcz_store.cc:269] 0709 13:18:57.019165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.019590 (+   425us) service_pool.cc:224] Handling call
0709 13:18:59.432150 (+2412560us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b540018f954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.436109 (+  3959us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4613982080}
I20260709 13:18:59.448117 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.448702 16587 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.449091 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.448623 17419 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:18:59.451376 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 123709us
W20260709 13:18:59.453088 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 142510us
I20260709 13:18:59.455909 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.448608 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.454136 19470 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.459765 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 605) took 1971 ms. Trace:
I20260709 13:18:59.463482 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.465302 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.460341 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 440) took 1002 ms. Trace:
I20260709 13:18:59.460053 17414 rpcz_store.cc:276] 0709 13:18:57.489816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.490039 (+   223us) service_pool.cc:224] Handling call
0709 13:18:57.506077 (+ 16038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.091147 (+1585070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.440681 (+349534us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54005d1bd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:59.459703 (+ 19022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3423571712}
I20260709 13:18:59.472494 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 1994) took 3082 ms. Trace:
I20260709 13:18:59.444278 19950 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:18:59.472867 16594 rpcz_store.cc:276] 0709 13:18:56.389886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.903303 (+513417us) service_pool.cc:224] Handling call
0709 13:18:58.838923 (+1935620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.139434 (+300511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.342337 (+202903us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:59.462554 (+120217us) spinlock_profiling.cc:243] Waited 1.93 s on lock 0x7b540010d450. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.472439 (+  9885us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4950443520}
W20260709 13:18:59.479977 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 113875us
W20260709 13:18:59.481590 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 170208us
I20260709 13:18:59.438333 20017 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.455240 20028 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.470455 16745 rpcz_store.cc:276] 0709 13:18:58.449774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:58.841404 (+391630us) service_pool.cc:224] Handling call
0709 13:18:58.841866 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.036233 (+194367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.036272 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.080040 (+ 43768us) log.cc:844] Serialized 26 byte log entry
0709 13:18:59.165093 (+ 85053us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b180036f880. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:18:59.259147 (+ 94054us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:59.259195 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.451605 (+192410us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400049054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:59.451692 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:59.451851 (+   159us) raft_consensus.cc:1685] finished
0709 13:18:59.451872 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.452052 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28014,"raft.queue_time_us":38704,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":44758,"spinlock_wait_cycles":376843264,"thread_start_us":93855,"threads_started":1,"wal-append.queue_time_us":180197}
I20260709 13:18:59.480499 16670 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:59.486371 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.399444 20003 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.489847 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:59.490273 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.490016 16667 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.489312 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:59.511040 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.512590 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.493314 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:59.492164 16736 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:59.510056 16666 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:18:59.516571 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:18:59.518936 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.486034 20036 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.524155 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.522830 16670 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:59.490432 20002 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:18:59.491717 20020 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:18:59.525355 19904 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:18:59.559343 20020 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:18:59.530988 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:59.531582 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 541) took 2571 ms. Trace:
I20260709 13:18:59.560508 20020 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.532990 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:59.561702 16744 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:59.534109 20009 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.534462 19577 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:18:59.534039 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48700.2us
I20260709 13:18:59.563954 20009 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:59.560945 17420 rpcz_store.cc:276] 0709 13:18:56.886574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.886835 (+   261us) service_pool.cc:224] Handling call
0709 13:18:57.008456 (+121621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:57.174545 (+166089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:57.174572 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:57.175000 (+   428us) log.cc:844] Serialized 26 byte log entry
0709 13:18:57.479985 (+304985us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:18:57.480045 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:57.480147 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:18:58.904550 (+1424403us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:18:59.084345 (+179795us) raft_consensus.cc:1685] finished
0709 13:18:59.084368 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.446322 (+361954us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540030cd50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.458284 (+ 11962us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8488,"raft.queue_time_us":414408,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":9342,"spinlock_wait_cycles":295906432,"thread_start_us":409115,"threads_started":2,"wal-append.queue_time_us":563815}
I20260709 13:18:59.534984 19861 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:18:59.535177 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.536223 20028 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:59.539021 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63079.6us
I20260709 13:18:59.560803 16741 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:59.541014 20045 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.541860 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.493440 19999 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.493666 20016 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:18:59.546439 20036 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.551468 19939 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.552925 19560 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.528504 20002 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:18:59.515846 19804 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.560768 16667 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: [5.4-6.5]
I20260709 13:18:59.534783 19860 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:18:59.539211 19881 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:18:59.575947 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 68063.7us
I20260709 13:18:59.571919 16591 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:59.576776 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:18:59.577251 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.587582 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1335) took 2558 ms. Trace:
I20260709 13:18:59.577841 16669 raft_consensus.cc:2368] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10.
I20260709 13:18:59.577987 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 2010) took 3112 ms. Trace:
I20260709 13:18:59.590480 16671 raft_consensus.cc:2410] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:18:59.588622 16590 rpcz_store.cc:276] 0709 13:18:56.465754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:56.939708 (+473954us) service_pool.cc:224] Handling call
0709 13:18:56.978110 (+ 38402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.365846 (+2387736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.515121 (+149275us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400235fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:18:59.577914 (+ 62793us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5480844032}
I20260709 13:18:59.588001 16667 rpcz_store.cc:276] 0709 13:18:57.024446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.559711 (+2535265us) service_pool.cc:224] Handling call
0709 13:18:59.560310 (+   599us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:18:59.582612 (+ 22302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.582923 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.578267 16593 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:59.578601 16736 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:59.595050 16663 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.577443 16587 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:18:59.581069 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:59.596510 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:59.597141 16744 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:59.598663 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.579896 19997 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.603107 16664 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.574994 19849 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:18:59.594434 20021 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:18:59.595408 20003 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:18:59.575197 19950 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.598140 19830 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:18:59.597681 19765 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.597735 20028 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.630874 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.612344 16670 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:59.628149 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:59.634954 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.635605 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:59.640178 20002 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.590932 19939 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.641742 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.624887 20021 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:59.630193 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:18:59.647919 16744 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:18:59.649441 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:59.652261 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.658357 16594 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:18:59.653244 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1336) took 2626 ms. Trace:
I20260709 13:18:59.653518 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1332) took 2628 ms. Trace:
I20260709 13:18:59.660403 16666 rpcz_store.cc:276] 0709 13:18:57.023554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.574589 (+2551035us) service_pool.cc:224] Handling call
0709 13:18:59.607291 (+ 32702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.647455 (+ 40164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.647482 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:18:59.651536 (+  4054us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:18:59.651583 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.651652 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:18:59.651748 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.661440 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.661983 16670 rpcz_store.cc:276] 0709 13:18:57.024705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.571675 (+2546970us) service_pool.cc:224] Handling call
0709 13:18:59.595744 (+ 24069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.650502 (+ 54758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.650711 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.662400 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.658370 16736 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:59.669710 16593 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:18:59.672283 17420 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:18:59.675442 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:18:59.680557 17374 connection.cc:256] RPC call timeout handler was delayed by -1.4196817110000666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260709 13:18:59.681183 17374 connection.cc:256] RPC call timeout handler was delayed by -1.4196817110000666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:18:59.660211 20021 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:18:59.676129 16667 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
W20260709 13:18:59.681609 17374 connection.cc:256] RPC call timeout handler was delayed by -1.2783817110000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 13:18:59.685222 17374 connection.cc:256] RPC call timeout handler was delayed by -1.2783817110000655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 13:18:59.685544 17374 connection.cc:256] RPC call timeout handler was delayed by -1.0155817110000953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
I20260709 13:18:59.683284 16668 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:18:59.683642 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:18:59.679574 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.653729 19822 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.709120 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.689278 19577 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.711544 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.719732 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:18:59.719179 16670 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:18:59.728415 17414 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 6: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:18:59.731426 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 128278us
I20260709 13:18:59.696806 20036 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.732708 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1338) took 2660 ms. Trace:
I20260709 13:18:59.733240 16670 rpcz_store.cc:276] 0709 13:18:57.061172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.662846 (+2601674us) service_pool.cc:224] Handling call
0709 13:18:59.702987 (+ 40141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.721412 (+ 18425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.721683 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.727319 19577 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.735872 16667 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:18:59.736481 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1958) took 2561 ms. Trace:
I20260709 13:18:59.736788 16667 rpcz_store.cc:276] 0709 13:18:57.174821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.685013 (+2510192us) service_pool.cc:224] Handling call
0709 13:18:59.735576 (+ 50563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.736246 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.736428 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.732391 17411 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:18:59.739184 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.704487 19758 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:18:59.745687 17416 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:18:59.746018 17420 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:18:59.741691 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:59.754987 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.755348 17410 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:18:59.755369 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.756276 16593 raft_consensus.cc:2410] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:18:59.717774 19470 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:18:59.749683 19577 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:18:59.763761 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.745222 19560 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.753997 20002 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:18:59.775300 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:59.776106 16744 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:18:59.768847 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.777711 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.747427 20036 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:18:59.770298 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:18:59.755048 20009 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:18:59.784898 17414 raft_consensus.cc:2410] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:18:59.778676 16667 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:59.783527 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:59.792482 16739 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:59.783447 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.783891 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:59.778256 17411 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:18:59.784195 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:18:59.767141 20026 consensus_meta.cc:287] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.003s
I20260709 13:18:59.794162 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:18:59.797516 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:18:59.797814 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44362.7us
I20260709 13:18:59.799372 16742 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.766953 19804 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.798812 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1341) took 2561 ms. Trace:
I20260709 13:18:59.801017 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.800922 16667 rpcz_store.cc:276] 0709 13:18:57.237388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.747564 (+2510176us) service_pool.cc:224] Handling call
0709 13:18:59.775826 (+ 28262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.798537 (+ 22711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.798761 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.801808 17416 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:18:59.801208 17420 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:18:59.824031 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:18:59.824818 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.851289 16670 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:18:59.858723 16594 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:59.858677 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:18:59.863348 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1340) took 2631 ms. Trace:
I20260709 13:18:59.864369 16670 rpcz_store.cc:276] 0709 13:18:57.231933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.744494 (+2512561us) service_pool.cc:224] Handling call
0709 13:18:59.782951 (+ 38457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.852069 (+ 69118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.863268 (+ 11199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.867942 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.792045 20009 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:18:59.875123 19837 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:18:59.876111 20044 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:18:59.879549 20062 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:18:59.880600 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.897957 16592 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:59.880927 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.882447 19560 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:18:59.886529 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1963) took 1376 ms. Trace:
I20260709 13:18:59.898847 16671 rpcz_store.cc:276] 0709 13:18:58.510035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.073468 (+563433us) service_pool.cc:224] Handling call
0709 13:18:59.886285 (+812817us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540009ea54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:18:59.886460 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186477952}
I20260709 13:18:59.889341 16663 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:18:59.891542 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 201629us
I20260709 13:18:59.898509 16593 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:59.899561 16667 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:18:59.900290 19837 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:18:59.900916 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.901314 16594 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:59.900926 16664 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:59.907944 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1966) took 1148 ms. Trace:
I20260709 13:18:59.908289 16664 rpcz_store.cc:276] 0709 13:18:58.759834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.900404 (+1140570us) service_pool.cc:224] Handling call
0709 13:18:59.900740 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.907663 (+  6923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.907856 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.903985 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1344) took 1728 ms. Trace:
I20260709 13:18:59.905503 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:18:59.910910 17411 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:18:59.902848 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.906322 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1965) took 1355 ms. Trace:
I20260709 13:18:59.909875 16667 rpcz_store.cc:276] 0709 13:18:58.175227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.801923 (+1626696us) service_pool.cc:224] Handling call
0709 13:18:59.899342 (+ 97419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.903675 (+  4333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.903921 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:18:59.904670 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.912081 16663 rpcz_store.cc:276] 0709 13:18:58.551253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.888697 (+1337444us) service_pool.cc:224] Handling call
0709 13:18:59.889125 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.906000 (+ 16875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.906254 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.890435 20021 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.917277 16592 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.893292 20009 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.919636 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.919695 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:18:59.947185 16593 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.921653 19837 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:18:59.922547 16671 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:59.923200 19822 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.901120 20037 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:59.931214 20028 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:18:59.917922 20018 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 13:18:59.936651 19582 consensus_meta.cc:287] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.003s
I20260709 13:18:59.938011 16664 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:18:59.961726 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1967) took 1201 ms. Trace:
I20260709 13:18:59.962154 16664 rpcz_store.cc:276] 0709 13:18:58.760390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.909036 (+1148646us) service_pool.cc:224] Handling call
0709 13:18:59.937660 (+ 28624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.961404 (+ 23744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.961658 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.939159 19810 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:18:59.803000 20026 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:18:59.796746 20036 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:18:59.942961 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.891889 19904 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:18:59.943672 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:18:59.946033 16588 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:18:59.944517 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:59.949606 16667 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:18:59.950292 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52697.8us
I20260709 13:18:59.951474 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.972702 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1968) took 1212 ms. Trace:
I20260709 13:18:59.972848 16593 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:18:59.952569 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1346) took 1189 ms. Trace:
I20260709 13:18:59.973188 16667 rpcz_store.cc:276] 0709 13:18:58.760834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.912406 (+1151572us) service_pool.cc:224] Handling call
0709 13:18:59.946067 (+ 33661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.972424 (+ 26357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.972637 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.973711 16671 rpcz_store.cc:276] 0709 13:18:58.795022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.914742 (+1119720us) service_pool.cc:224] Handling call
0709 13:18:59.922366 (+  7624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.952206 (+ 29840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.952501 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.955782 16670 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:18:59.956199 16591 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:18:59.977943 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1345) took 1533 ms. Trace:
I20260709 13:18:59.978456 16670 rpcz_store.cc:276] 0709 13:18:58.444483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.892340 (+1447857us) service_pool.cc:224] Handling call
0709 13:18:59.955613 (+ 63273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.977576 (+ 21963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.977857 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.954342 16736 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
W20260709 13:18:59.954823 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44767.4us
I20260709 13:18:59.960522 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.965536 16663 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:18:59.983350 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44468.6us
I20260709 13:18:59.984405 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:18:59.985014 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.994978 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 1969) took 1233 ms. Trace:
I20260709 13:18:59.995491 16663 rpcz_store.cc:276] 0709 13:18:58.761242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.913177 (+1151935us) service_pool.cc:224] Handling call
0709 13:18:59.949235 (+ 36058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.988644 (+ 39409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:18:59.994825 (+  6181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:18:59.997567 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:18:59.964814 19582 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.008323 16670 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:00.002278 16592 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:18:59.919044 20044 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.003937 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.004874 16595 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:00.006300 17374 leader_election.cc:336] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.824s (SENT)
I20260709 13:19:00.011711 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:18:59.994113 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:00.024154 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:18:59.993397 20038 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:00.030037 16663 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:00.023687 19577 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:00.031987 17374 leader_election.cc:336] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.824s (SENT)
I20260709 13:19:00.032819 19577 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:00.033669 19577 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:00.035077 19996 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:00.035593 19996 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:00.036731 19996 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.038331 19996 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.043341 20018 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:00.045845 16671 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:18:59.992156 20037 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.053755 20037 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:00.013260 19911 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:00.007181 19810 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:19:00.056192 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.077s (SENT)
I20260709 13:19:00.034463 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:00.030457 20038 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.068141 20038 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:00.069259 20044 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:00.072656 16668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1322) took 3053 ms (client timeout 2198 ms). Trace:
I20260709 13:19:00.073367 20026 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:00.073843 20021 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:00.075205 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68633.9us
I20260709 13:19:00.076375 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:18:59.988267 19904 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:00.073091 16668 rpcz_store.cc:269] 0709 13:18:57.018798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:57.019510 (+   712us) service_pool.cc:224] Handling call
0709 13:19:00.072463 (+3052953us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b540001a754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:00.072588 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5706234112}
I20260709 13:19:00.079249 19810 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:00.093055 19810 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.093246 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.094004 19904 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:00.080364 19560 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:00.095669 19560 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:00.096918 16668 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:00.098366 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.099212 16665 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:00.100219 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.096939 19560 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.101840 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.103200 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.103539 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.103788 16590 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:00.104710 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.105142 19830 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:00.081897 19999 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:00.106050 19999 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:00.106256 19830 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:00.107312 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.107182 19999 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.108065 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:00.108683 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.108595 19999 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.109373 16670 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:00.084184 19950 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:00.085494 19837 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:00.090991 20036 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:00.092084 20064 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:19:00.105579 16590 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:19:00.081179 19758 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:00.110489 16592 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:00.104710 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:00.103215 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.094658 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:00.113169 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 78360.3us
I20260709 13:19:00.114692 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:00.121061 16593 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:19:00.121692 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:00.124425 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.136806 16736 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:19:00.062733 19911 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.130378 16665 raft_consensus.cc:2376] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9. Re-sending same reply.
I20260709 13:19:00.124440 19837 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:00.139891 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:00.136569 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:00.150074 16590 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.147818 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.140599 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.148788 16634 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:00.150974 16667 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:19:00.106762 19830 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:00.152153 16594 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.154414 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.155840 16670 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:00.156415 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.158591 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:00.159914 20002 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:00.160573 20002 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:00.161911 20017 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.161614 20002 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.164386 19939 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:00.164841 16634 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:00.165091 19851 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:00.165977 19804 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:00.166428 19804 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.166828 19804 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:00.167516 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.168121 16667 raft_consensus.cc:2376] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9. Re-sending same reply.
I20260709 13:19:00.169080 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.169950 16595 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:00.166719 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.171011 19851 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:00.155913 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.171609 16592 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:00.171988 16663 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:00.171850 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50302.7us
I20260709 13:19:00.146919 19911 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:00.172869 16592 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:19:00.173394 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:00.174935 20026 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:00.175005 20028 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:00.175573 20026 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:00.176597 20026 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.135339 19758 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.183578 16588 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:00.179843 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:00.181731 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.187470 16593 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:00.188383 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.188689 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:00.189611 16589 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:00.190809 16591 raft_consensus.cc:2368] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:19:00.151741 20021 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:00.191006 19582 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:19:00.224146 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65064.2us
I20260709 13:19:00.222846 16590 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:00.219077 16593 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:00.193015 16665 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:00.172057 19851 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.232780 16591 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:00.234685 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1339) took 2998 ms. Trace:
I20260709 13:19:00.150122 19950 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 13:19:00.240722 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:00.218320 20009 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.220201 20021 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.242708 16596 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:00.252493 16742 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 13:19:00.238003 20003 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.004s
I20260709 13:19:00.253037 16742 pending_rounds.cc:85] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:19:00.235275 16666 rpcz_store.cc:276] 0709 13:18:57.231671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.706331 (+2474660us) service_pool.cc:224] Handling call
0709 13:18:59.743163 (+ 36832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:18:59.763648 (+ 20485us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:18:59.797542 (+ 33894us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.229893 (+432351us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.229954 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:00.230023 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.230112 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.230302 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":218,"prepare.queue_time_us":33866,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":58,"raft.queue_time_us":5164,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":335252,"threads_started":1}
I20260709 13:19:00.239692 19939 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:19:00.246311 20021 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.272955 16662 raft_consensus.cc:2410] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:19:00.274327 16594 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:00.256992 16742 pending_rounds.cc:107] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:19:00.276607 16742 raft_consensus.cc:2889] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:00.280313 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 1954) took 3209 ms. Trace:
I20260709 13:19:00.280751 16662 rpcz_store.cc:276] 0709 13:18:57.070648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.448797 (+2378149us) service_pool.cc:224] Handling call
0709 13:19:00.280114 (+831317us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54003890d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:00.280249 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888136960}
I20260709 13:19:00.273523 19582 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:00.316800 19582 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.333774 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.334486 16742 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.318162 19549 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:00.345634 20067 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:00.338977 20003 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.348697 20003 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.275192 19837 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.274363 19830 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:00.387857 20020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.015s	sys 0.006s
W20260709 13:19:00.388429 20020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.015s	sys 0.006s
I20260709 13:19:00.389772 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1351) took 1401 ms. Trace:
I20260709 13:19:00.390267 16669 rpcz_store.cc:276] 0709 13:18:58.988326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.963781 (+975455us) service_pool.cc:224] Handling call
0709 13:18:59.975864 (+ 12083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.031544 (+ 55680us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:00.031844 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.389435 (+357591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.389490 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:00.389532 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.389598 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.389717 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":612,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":104,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":356905,"threads_started":1}
W20260709 13:19:00.408227 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.409241 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.346556 19549 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:00.413837 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 382147us
I20260709 13:19:00.422353 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.426424 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.429509 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.438709 16739 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.455951 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.373150 19911 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.435082 19830 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.428288 20003 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:00.463150 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.466430 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.464669 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.467808 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.468550 16741 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:00.439695 20009 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:00.467355 16737 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:19:00.471752 16708 leader_election.cc:336] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.615s (SENT)
I20260709 13:19:00.462800 19837 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:00.480588 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.485026 16740 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:00.487972 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.497514 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.498382 16740 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.500617 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:00.505765 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.516990 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.518210 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46475.2us
W20260709 13:19:00.518683 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 367309us
I20260709 13:19:00.523244 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.478332 19804 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:00.524802 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 490369us
W20260709 13:19:00.526772 16636 connection.cc:256] RPC call timeout handler was delayed by -1.2690341439999884s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:19:00.525372 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 494155us
W20260709 13:19:00.527912 16636 connection.cc:256] RPC call timeout handler was delayed by -1.1169341439999698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:19:00.528517 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 288244us
W20260709 13:19:00.528344 16636 connection.cc:256] RPC call timeout handler was delayed by -1.0701341439998941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260709 13:19:00.529489 16636 connection.cc:256] RPC call timeout handler was delayed by -1.0035341440000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1836s for a response
I20260709 13:19:00.527629 16739 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:19:00.529640 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 288968us
I20260709 13:19:00.518462 20028 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:00.541556 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:00.546072 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:00.547709 16594 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:00.549803 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.552408 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.543954 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:00.557145 16559 leader_election.cc:400] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10.
W20260709 13:19:00.556299 16708 leader_election.cc:336] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.758s (SENT)
I20260709 13:19:00.557572 16559 leader_election.cc:403] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:00.543100 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.558724 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.559554 16739 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:00.556280 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.487279 19549 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:00.573168 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.577769 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.524048 20038 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.535156 20067 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:19:00.584399 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.597002 16744 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:00.602265 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.602670 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.603135 16664 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:00.611718 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.612658 16667 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:00.603292 16663 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
W20260709 13:19:00.632817 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.642901 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.643463 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.643661 16662 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:00.651628 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:00.652982 16666 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:19:00.656427 20075 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:19:00.659358 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:00.613981 19810 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.000s
I20260709 13:19:00.660471 16671 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:00.682191 16742 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:00.687067 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:00.688491 19470 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:00.689213 19470 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:00.671939 16668 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:00.687228 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:00.691614 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 135331us
I20260709 13:19:00.612329 20091 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.049s
W20260709 13:19:00.701793 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 164040us
W20260709 13:19:00.696187 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.669173 20070 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
W20260709 13:19:00.703228 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.704134 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:00.601290 19765 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.070s
W20260709 13:19:00.724292 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 182374us
W20260709 13:19:00.725018 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.756270 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 199382us
I20260709 13:19:00.764634 20045 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:00.771450 20045 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:00.773103 20080 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:00.741179 20071 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:19:00.780217 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.782302 16742 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:00.690670 19810 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.773639 20080 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:00.735193 19939 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:00.726949 19950 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:19:00.739661 19911 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:00.739903 19765 consensus_queue.cc:1243] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:19:00.796484 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:00.802469 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:00.784579 17376 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:00.818783 16738 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:00.823804 17419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 611) took 1217 ms. Trace:
I20260709 13:19:00.824318 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:00.825462 16742 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:00.824219 17419 rpcz_store.cc:276] 0709 13:18:59.608082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.624388 (+ 16306us) service_pool.cc:224] Handling call
0709 13:18:59.632198 (+  7810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.671488 (+ 39290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.671520 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.701411 (+ 29891us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.227956 (+526545us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:00.227995 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.228078 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:00.823567 (+595489us) raft_consensus.cc:1685] finished
0709 13:19:00.823587 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.823747 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6835,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3283,"thread_start_us":420137,"threads_started":1,"wal-append.queue_time_us":644198}
I20260709 13:19:00.795248 20045 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:19:00.828138 20045 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:00.828429 20080 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.830483 20080 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:00.831648 20091 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:19:00.832535 20096 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.182s	user 0.001s	sys 0.001s
I20260709 13:19:00.795686 20097 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10.
W20260709 13:19:00.836223 20074 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 13:19:00.836997 20073 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:19:00.796716 20028 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.839543 20082 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.907234 20082 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:00.907914 20077 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
I20260709 13:19:00.910207 20082 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:00.840394 20038 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:00.828496 20045 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:00.852845 20093 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:00.848742 16740 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:00.852661 19810 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:00.875445 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 111794us
I20260709 13:19:00.742257 19470 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:00.879338 19904 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.889472 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2689) took 1003 ms. Trace:
I20260709 13:19:00.851069 17376 leader_election.cc:403] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:00.760736 20067 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:00.896776 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:00.901226 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:00.904732 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 147413us
I20260709 13:19:00.950109 17415 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:00.950225 17419 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:00.905145 19911 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:00.906402 16744 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:19:00.847508 20021 consensus_meta.cc:287] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.000s
I20260709 13:19:00.885193 16741 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:00.944151 16592 rpcz_store.cc:276] 0709 13:18:59.886160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.173696 (+287536us) service_pool.cc:224] Handling call
0709 13:19:00.173986 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.174196 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:00.532833 (+358637us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.739054 (+206221us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.889014 (+149960us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.889088 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:00.889144 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.889250 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:00.889398 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13080,"prepare.queue_time_us":665926,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":76,"raft.queue_time_us":198549,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":684328,"threads_started":3}
W20260709 13:19:00.948110 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 190808us
W20260709 13:19:00.991320 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:00.997727 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56287.6us
I20260709 13:19:00.862962 20104 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 13:19:01.001051 16742 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:19:01.007629 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:00.924005 20091 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:19:00.955426 20097 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:01.013561 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.995s (SENT)
I20260709 13:19:00.934974 20020 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:19:00.946172 20036 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:01.016553 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:00.923192 20018 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:00.997061 20082 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.019829 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:00.993000 19911 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:01.022619 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.077s (SENT)
I20260709 13:19:01.023103 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.023168 19810 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:01.029892 17419 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:01.019963 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:01.033712 17413 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:00.950608 19904 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.034731 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 622) took 1129 ms. Trace:
I20260709 13:19:01.030805 19911 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:01.025614 19996 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.987s	user 0.008s	sys 0.008s
W20260709 13:19:01.040257 17374 leader_election.cc:336] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.166s (SENT)
W20260709 13:19:01.040400 20094 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:19:01.040742 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:01.041977 17374 leader_election.cc:336] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.197s (SENT)
W20260709 13:19:01.042131 16708 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.877s (SENT)
I20260709 13:19:01.042562 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:01.042928 17374 leader_election.cc:336] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.226s (SENT)
I20260709 13:19:01.040537 19996 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.044677 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:01.045282 17412 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:01.045594 19804 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:01.046252 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:01.046970 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.047199 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2690) took 1160 ms. Trace:
I20260709 13:19:01.047562 16588 rpcz_store.cc:276] 0709 13:18:59.886408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.194450 (+308042us) service_pool.cc:224] Handling call
0709 13:19:00.194959 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.229008 (+ 34049us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.229051 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.257829 (+ 28778us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.699041 (+441212us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:00.699085 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.699160 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.046990 (+347830us) raft_consensus.cc:1685] finished
0709 13:19:01.047013 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.047141 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128114,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":8205,"thread_start_us":340925,"threads_started":1,"wal-append.queue_time_us":641785}
I20260709 13:19:01.048364 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:01.049856 17420 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:01.051525 20106 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:19:01.052876 19577 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.053365 19577 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.046802 17412 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.055365 19577 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.035128 17414 rpcz_store.cc:276] 0709 13:18:59.909104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.918071 (+  8967us) service_pool.cc:224] Handling call
0709 13:18:59.923195 (+  5124us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:18:59.923607 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:18:59.923638 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:18:59.924143 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.330841 (+406698us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:00.332859 (+  2018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.332972 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.034528 (+701556us) raft_consensus.cc:1685] finished
0709 13:19:01.034547 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.034675 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8522,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"thread_start_us":406002,"threads_started":1,"wal-append.queue_time_us":591048}
I20260709 13:19:01.056439 19577 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:01.057443 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:01.036847 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 294716us
I20260709 13:19:01.058593 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:00.991818 20021 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.017096 19549 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.063493 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:01.063794 17413 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:01.065651 17412 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:01.015714 20097 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:01.020730 20036 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:01.031414 20045 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.083168 17414 raft_consensus.cc:2410] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:19:01.084445 20087 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:01.084857 20087 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:01.086898 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.088891 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:01.039970 20078 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:01.087152 19765 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:01.097712 20028 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:01.089988 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:01.091571 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.101286 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:01.101895 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 172922us
W20260709 13:19:01.104411 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.105152 17415 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:01.035712 19822 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:19:01.086457 20088 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.061300 17419 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:19:01.071231 20082 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:01.092553 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 72717.7us
W20260709 13:19:01.115278 19582 consensus_meta.cc:287] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.792s	user 0.010s	sys 0.003s
I20260709 13:19:01.105414 19804 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:01.116492 17412 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:01.081147 20098 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:19:01.119560 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:01.085788 20087 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.122427 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:01.123189 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47791.1us
I20260709 13:19:01.123339 20045 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.124884 16594 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:01.112854 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:01.120199 19758 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.895s	user 0.005s	sys 0.009s
I20260709 13:19:01.109608 19765 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.119227 17419 pending_rounds.cc:85] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 5
I20260709 13:19:01.132315 16587 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:01.132725 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:01.150197 16589 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.149516 17419 pending_rounds.cc:107] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:19:01.150985 17419 raft_consensus.cc:2889] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:01.155887 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2608) took 1044 ms. Trace:
I20260709 13:19:01.156445 16738 rpcz_store.cc:276] 0709 13:19:00.111480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.111683 (+   203us) service_pool.cc:224] Handling call
0709 13:19:00.136036 (+ 24353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.822229 (+686193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.155589 (+333360us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54003ee9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:01.155814 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569485568}
I20260709 13:19:01.121905 19996 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.107231 19549 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.153801 19765 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.125241 20036 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.176338 20036 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:01.186378 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:01.187520 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 74737.9us
W20260709 13:19:01.189081 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 66710.2us
I20260709 13:19:01.180558 19950 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.198221 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:01.131557 19582 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.133900 20093 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
W20260709 13:19:01.196053 20103 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.175s	user 0.001s	sys 0.001s
W20260709 13:19:01.196869 19837 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.004s
I20260709 13:19:01.166071 19758 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.216615 16589 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:01.134235 20097 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.208434 19950 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:01.233642 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97478.9us
I20260709 13:19:01.224965 20128 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:01.229048 19837 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.252779 20026 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:01.227226 20121 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.234028 20021 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:01.235536 19904 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:01.279958 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53332.5us
I20260709 13:19:01.284395 16668 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: [4.2-7.4]
I20260709 13:19:01.288975 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:01.289777 16665 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:01.291994 20110 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:19:01.293213 19911 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:19:01.281471 20115 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.184s	user 0.003s	sys 0.000s
I20260709 13:19:01.294780 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:01.298301 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 100153us
I20260709 13:19:01.267946 20026 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:01.296983 16666 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:01.300248 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.279425 19837 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:01.277946 20128 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:01.303289 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.304243 16668 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:01.304411 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.299346 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:01.286017 20097 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.297461 19911 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:01.301100 16662 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:01.319197 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:01.320004 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:01.326332 16671 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.322149 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.333829 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:01.337862 16662 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:01.296277 16592 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73089175ee9c465197a1a11e304346be"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:19:01.340487 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:01.340853 16592 raft_consensus.cc:606] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.341289 16666 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:01.341364 16592 raft_consensus.cc:614] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:01.344939 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57231.6us
I20260709 13:19:01.346053 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.318594 19911 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.350533 16670 pending_rounds.cc:85] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 3
I20260709 13:19:01.351017 16670 pending_rounds.cc:107] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:19:01.351357 16670 raft_consensus.cc:2889] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:01.319391 20124 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:19:01.311606 20083 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:19:01.320936 20067 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:01.354331 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.361806 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:01.362488 16662 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:01.315677 19582 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:01.347913 19549 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:01.336751 20121 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:01.306110 20127 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:01.320236 20082 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:01.366142 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.348805 19911 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:01.371006 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48969.6us
W20260709 13:19:01.372473 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 65512.1us
W20260709 13:19:01.394308 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.379618 16665 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
W20260709 13:19:01.382187 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.360555 20067 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:01.400317 16742 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:01.400383 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 195828us
I20260709 13:19:01.406299 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.407677 16664 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:01.370960 20009 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.415904 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.409301 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:01.315160 19950 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.316421 19837 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.418238 16739 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:01.423725 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:01.427587 16738 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
W20260709 13:19:01.432152 17374 leader_election.cc:336] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.557s (SENT)
I20260709 13:19:01.432858 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.391650 20121 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:01.408353 20080 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:01.319871 20026 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:01.453848 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.434172 20003 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:01.434983 20045 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:01.393139 20127 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.409176 20082 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:19:01.459149 20003 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:01.461694 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45931.5us
I20260709 13:19:01.474522 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:01.444638 19758 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.413863 20078 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:19:01.450525 20009 raft_consensus.cc:2764] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:19:01.457223 20098 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:01.485546 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 69849.7us
W20260709 13:19:01.486735 20130 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:01.477818 19810 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:19:01.509545 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:01.509850 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:01.494410 16662 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
W20260709 13:19:01.503278 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.510470 17416 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:01.477780 20003 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:01.512202 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:01.528002 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:01.547363 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.440923 20128 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.536031 20026 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.539445 20121 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:01.541996 19830 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.074s	user 0.006s	sys 0.006s
I20260709 13:19:01.542681 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:01.543457 20140 raft_consensus.cc:2764] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
W20260709 13:19:01.539399 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.547433 17415 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:19:01.528520 20127 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.562752 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:01.572026 17412 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:01.584106 16591 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:01.585412 16666 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
W20260709 13:19:01.586431 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 146867us
W20260709 13:19:01.560189 20136 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:19:01.569979 20126 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:19:01.591881 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88185.3us
I20260709 13:19:01.594373 16708 leader_election.cc:400] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:19:01.594651 16708 leader_election.cc:403] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:19:01.595497 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:01.596304 20099 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:01.606433 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.590965 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 618) took 1877 ms. Trace:
W20260709 13:19:01.627805 20117 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.483s	user 0.003s	sys 0.000s
I20260709 13:19:01.626958 17409 rpcz_store.cc:276] 0709 13:18:59.718147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.730818 (+ 12671us) service_pool.cc:224] Handling call
0709 13:18:59.752962 (+ 22144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:00.079082 (+326120us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:00.519915 (+440833us) log.cc:844] Serialized 14 byte log entry
0709 13:19:00.851454 (+331539us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:00.895142 (+ 43688us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:00.895501 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:01.582751 (+687250us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:01.585473 (+  2722us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.589507 (+  4034us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":480525,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":36479,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":85,"spinlock_wait_cycles":581717376,"thread_start_us":642055,"threads_started":2,"wal-append.queue_time_us":311852}
W20260709 13:19:01.630352 20095 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.529s	user 0.000s	sys 0.002s
I20260709 13:19:01.597021 19765 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:01.632820 20084 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.489s	user 0.001s	sys 0.001s
I20260709 13:19:01.633623 19765 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:01.635099 20151 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.635327 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2688) took 1749 ms. Trace:
I20260709 13:19:01.601516 20092 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.597640 19911 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:19:01.637182 17412 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:01.639001 16739 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:01.635793 16595 rpcz_store.cc:276] 0709 13:18:59.885857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.172821 (+286964us) service_pool.cc:224] Handling call
0709 13:19:00.173141 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.173389 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.173414 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.173878 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.517132 (+343254us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:00.517179 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.517452 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.631986 (+1114534us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:01.634974 (+  2988us) raft_consensus.cc:1685] finished
0709 13:19:01.635016 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.635261 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5873920,"thread_start_us":342843,"threads_started":1,"wal-append.queue_time_us":935706}
W20260709 13:19:01.624225 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.995s (SENT)
I20260709 13:19:01.651674 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.651913 20082 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.652951 20082 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.653766 20036 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:01.643462 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55404.6us
I20260709 13:19:01.654757 20036 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:01.655457 20082 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:01.657928 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.664875 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.667361 16671 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:01.635757 20151 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:01.666927 20128 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:19:01.666599 20140 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:19:01.688827 20151 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.673995 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:01.690307 16739 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:01.691547 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:01.685165 20003 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.693843 20003 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:01.699769 20146 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:19:01.721623 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48593.8us
W20260709 13:19:01.670516 20142 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
W20260709 13:19:01.738853 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74084.6us
I20260709 13:19:01.702467 20093 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:01.721330 20018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.007s	sys 0.029s
I20260709 13:19:01.747036 20127 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:01.747725 20127 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.749943 20127 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.750537 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2609) took 1528 ms. Trace:
I20260709 13:19:01.750953 16736 rpcz_store.cc:276] 0709 13:19:00.222705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:00.222889 (+   184us) service_pool.cc:224] Handling call
0709 13:19:00.231716 (+  8827us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:00.232212 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:00.232255 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:00.239609 (+  7354us) log.cc:844] Serialized 26 byte log entry
0709 13:19:00.516672 (+277063us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:00.516729 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:00.516826 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:01.629608 (+1112782us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:01.742491 (+112883us) raft_consensus.cc:1685] finished
0709 13:19:01.742514 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:01.750468 (+  7954us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":996,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4970,"spinlock_wait_cycles":102411520,"thread_start_us":243618,"threads_started":1,"wal-append.queue_time_us":763894}
I20260709 13:19:01.752840 20127 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:01.753484 20127 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.755620 20127 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.760088 20127 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:01.760679 20127 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.762960 20127 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.721889 20036 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:01.667434 20092 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:01.768874 20018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.008s	sys 0.029s
I20260709 13:19:01.783850 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.768631 20093 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:01.770789 20036 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:01.820554 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.816454 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.822139 20080 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.822528 20036 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.820791 20093 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.828068 20018 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.883455 20087 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.855158 19911 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:01.889793 20120 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.618s	user 0.000s	sys 0.006s
I20260709 13:19:01.889832 19911 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.893600 19911 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.856904 20067 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.894345 20067 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.857520 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.859995 16736 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:01.860670 20160 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:19:01.861591 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 565559us
W20260709 13:19:01.899289 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.863668 19999 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:01.863940 20155 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.208s	user 0.007s	sys 0.004s
W20260709 13:19:01.863874 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 517909us
W20260709 13:19:01.864727 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:01.872548 20092 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:01.821844 20003 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:01.881330 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.881577 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:01.881582 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65144us
I20260709 13:19:01.907709 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:01.909008 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.877357 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:01.888336 17374 leader_election.cc:336] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.632s (SENT)
I20260709 13:19:01.896435 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.903851 16559 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:01.911291 16663 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:01.911273 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.911657 16671 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:01.915567 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:01.913543 16634 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:01.903342 20119 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.008s	sys 0.016s
I20260709 13:19:01.919929 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:01.920272 17374 leader_election.cc:336] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.664s (SENT)
I20260709 13:19:01.921419 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:01.922578 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:01.923022 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:01.903663 19560 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.923815 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:01.926250 17374 leader_election.cc:336] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.664s (SENT)
I20260709 13:19:01.926232 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:01.926450 16591 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:01.927132 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:01.926826 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:01.930444 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:01.931452 16594 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:01.885769 20080 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.922917 19999 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.913384 20128 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:01.916894 20132 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:01.917102 20151 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:19:01.923699 20119 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.009s	sys 0.016s
I20260709 13:19:01.933634 20162 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:19:01.916311 20114 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:01.937738 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:01.922209 20026 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:01.928411 20113 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:01.939065 20104 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:01.924506 20028 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:01.923870 20127 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:01.929181 19810 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:01.942474 20086 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:01.916489 20092 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:19:01.919535 20018 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:19:01.943823 20148 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:01.951336 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.937071 20128 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.924186 20038 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:19:01.952629 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.954569 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.956310 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:01.942338 19577 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:01.965229 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:01.969521 16664 raft_consensus.cc:2410] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:19:01.955953 19810 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.943305 19999 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.969226 16670 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:01.973748 20028 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:01.975036 20148 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:01.974802 20028 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.975965 20148 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.967002 20097 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:19:01.978410 19810 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.956372 20086 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:01.979614 20086 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:01.971643 20104 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:01.953471 20127 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:01.984529 20127 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:01.984651 16671 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:01.984966 20104 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.987084 20113 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:01.987529 20113 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:01.987941 20113 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:01.991433 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:19:01.995366 20156 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:19:01.995549 19837 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:01.949698 20114 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:01.996169 19837 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:01.998129 20114 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:01.998369 19837 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:01.967626 20112 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:01.999984 16590 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:19:01.998766 16587 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:02.019358 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:01.999125 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.202s (SENT) [suppressed 39 similar messages]
I20260709 13:19:02.024418 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.024601 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.025476 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:02.026072 16662 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 13:19:02.026090 16663 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:02.027493 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.028316 16739 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:02.028378 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:02.029101 16736 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:02.029282 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.029418 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.029982 16740 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:02.031585 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.032172 16737 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:02.033561 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.034274 16737 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 5: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:02.034484 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:01.968420 20128 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.044207 16740 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:02.038770 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 47961.3us
I20260709 13:19:02.050714 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.051409 16596 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:01.970768 20026 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:02.053390 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:02.054083 16671 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:01.998729 20036 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.055792 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:02.001273 19999 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:02.063390 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.063915 16739 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:02.067130 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.067417 16595 raft_consensus.cc:2393] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:02.065254 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:02.067653 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.068387 16587 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:02.024596 20080 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.072465 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.074852 20018 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.078265 16740 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:19:02.045483 20114 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.048079 20087 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.081748 19999 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:02.085419 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.085613 16664 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:19:02.093858 20158 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.368s	user 0.000s	sys 0.007s
W20260709 13:19:02.097767 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.821s (SENT)
W20260709 13:19:02.127920 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 72197.9us
W20260709 13:19:02.128383 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.821s (SENT)
I20260709 13:19:02.128680 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.065827 19577 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:02.102586 20026 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.106102 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:02.105660 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:02.139298 16668 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:02.112653 20127 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:02.113646 20168 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 13:19:02.113790 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 217447us
I20260709 13:19:02.047993 20093 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.114706 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:02.115329 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 201834us
I20260709 13:19:02.115823 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.115988 20092 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:02.117003 16588 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:02.117856 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.174325 16594 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:02.118933 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:02.150919 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.121052 16595 raft_consensus.cc:2410] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:19:02.128768 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:02.175645 16636 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:02.178242 16636 leader_election.cc:403] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:02.130654 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.110422 20018 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.140528 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.127611 19549 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:02.138592 17409 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:02.182217 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.182930 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.183099 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:19:02.183933 16594 raft_consensus.cc:2393] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:02.100214 19810 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.101119 20080 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.179241 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50881.6us
I20260709 13:19:02.145043 20017 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.189658 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.190436 16591 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.185179 16589 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:02.128010 19999 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.185945 16739 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:02.187239 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56697.9us
I20260709 13:19:02.187393 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.142409 20003 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:02.144832 19560 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:02.188448 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.194509 16593 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:02.195063 16591 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:02.149935 19577 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.066329 20028 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.110s
I20260709 13:19:02.193041 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.171682 20035 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.145756 20087 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.203137 20035 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.204995 20087 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.205370 20035 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:02.193077 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.207597 20036 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:02.175944 20092 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:02.209937 20093 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:02.209645 20092 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.211119 20092 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.148389 20118 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:19:02.215104 20140 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:02.215515 20140 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:02.215586 20118 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:02.216066 20118 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:02.216349 20140 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.216998 20118 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.218325 19549 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:02.193724 20080 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.219044 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.219763 16588 raft_consensus.cc:2368] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:02.219447 19549 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.181262 20082 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.220628 20080 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.223321 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.224526 20020 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:02.224740 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:02.225392 16740 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.225559 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:02.225919 19577 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:02.193962 16663 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.226442 16743 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:02.228186 19810 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:02.228534 20003 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.197458 16595 raft_consensus.cc:2376] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8. Re-sending same reply.
I20260709 13:19:02.226467 17409 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:02.195601 20038 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:19:02.251539 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.198880 20165 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:19:02.224920 20020 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.226226 17415 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:02.230234 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:02.264813 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.188740 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:02.193257 17374 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.886s (SENT)
I20260709 13:19:02.265981 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.236554 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:02.123805 20086 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.245348 16740 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:02.206638 20127 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.280359 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.267414 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:02.281808 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.270560 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:02.283792 16668 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.194607 19999 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:02.284399 19904 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:02.284754 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.285512 16595 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:02.189868 20017 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:02.285818 19804 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.286099 20017 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.272862 20020 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:02.195122 20099 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:02.294129 19560 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:02.294648 19560 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:02.280264 20036 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:02.295116 19758 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:02.295563 19758 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:02.296419 19758 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.296964 20086 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.297694 19758 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:02.300232 16592 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:02.275058 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:02.277433 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88776.5us
I20260709 13:19:02.218356 20118 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:02.302239 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109003us
I20260709 13:19:02.282467 16737 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:02.259464 16596 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:19:02.301450 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:02.284554 19470 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:02.246970 20038 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:02.309960 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.956s (SENT)
W20260709 13:19:02.403180 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 93245us
W20260709 13:19:02.403923 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.956s (SENT)
I20260709 13:19:02.404369 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:02.310500 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:02.405453 16663 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:02.256654 19810 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:19:02.381336 16666 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:02.409675 16666 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:02.406827 20038 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:02.408329 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.269317 20160 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
W20260709 13:19:02.413955 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 110222us
I20260709 13:19:02.394300 19837 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:02.409610 20067 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.003s
I20260709 13:19:02.422446 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:02.380292 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 68152.8us
W20260709 13:19:02.424316 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.429411 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 128022us
I20260709 13:19:02.430256 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:02.432420 17374 leader_election.cc:336] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.113s (SENT)
I20260709 13:19:02.417558 16666 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.433197 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:02.422519 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 139894us
I20260709 13:19:02.435987 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.441718 16666 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:02.446308 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.447355 16668 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:02.418434 20098 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:02.448355 17414 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:02.448655 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.449355 16596 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:02.450310 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:02.452195 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.453284 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.453896 16587 raft_consensus.cc:2376] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5. Re-sending same reply.
I20260709 13:19:02.431237 20088 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.436211 20067 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.462813 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.439735 20093 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:02.434325 20026 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:19:02.449641 20036 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:02.451340 20125 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.468923 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.420791 20099 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:19:02.469700 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.466543 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.451246 20086 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.473986 16666 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:02.477098 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.479462 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.479202 16595 raft_consensus.cc:2376] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8. Re-sending same reply.
I20260709 13:19:02.483529 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.485574 16588 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 6: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:02.491393 20172 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:19:02.495009 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.455374 19950 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:19:02.499534 16494 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 9, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 9 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:02.501369 16494 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:02.461382 20082 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:19:02.491417 20176 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:19:02.508064 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.484413 19804 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:02.461081 20182 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.507920 16494 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:02.518465 16591 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:19:02.472266 20127 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:02.472882 20067 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.488070 20036 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:19:02.508867 20179 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:19:02.470651 20088 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.524333 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.525816 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.530465 16593 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:02.525806 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:02.526858 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.487540 20086 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.521394 20182 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:02.541432 20036 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.543268 20088 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:02.545379 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:02.555809 20177 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:19:02.564253 20067 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:02.564915 20067 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:02.565430 20067 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:02.536173 20175 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
I20260709 13:19:02.567572 20017 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.539127 20182 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.512306 20128 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:02.570406 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 155177us
I20260709 13:19:02.604592 20086 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:02.606586 20113 consensus_meta.cc:287] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.618s	user 0.006s	sys 0.003s
I20260709 13:19:02.610136 16634 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:02.633325 16741 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:02.638415 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.618324 20099 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:02.662012 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.601015 19999 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:02.663371 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.615275 20166 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:19:02.609035 20020 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.663239 20114 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.614122 20113 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.689553 16589 raft_consensus.cc:2368] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
I20260709 13:19:02.671394 20099 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.699496 16736 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:02.715520 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:02.719897 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.720682 16736 raft_consensus.cc:2368] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:19:02.720888 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.721514 16743 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:19:02.721589 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:02.721843 20017 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:02.707427 20099 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:02.724843 20113 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.726348 20067 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:02.702096 20098 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:02.727284 20098 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.729365 20098 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:02.730706 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:02.731061 20020 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:02.731650 16737 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 7: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:02.732618 16739 pending_rounds.cc:85] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 3
I20260709 13:19:02.732900 16739 pending_rounds.cc:107] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:19:02.733122 16739 raft_consensus.cc:2889] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:02.744911 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.752095 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:02.752213 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:02.752904 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 126896us
I20260709 13:19:02.761823 16743 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:02.679242 20093 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:02.766065 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:02.707690 20185 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.760118 20147 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:19:02.787209 16593 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 13:19:02.790136 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 334996us
W20260709 13:19:02.792549 17374 leader_election.cc:336] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.289s (SENT)
W20260709 13:19:02.799407 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.768520 20038 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.817131 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.817888 16744 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:02.808765 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.822237 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.715165 20114 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:02.828677 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:02.823801 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:02.824841 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.829922 16737 raft_consensus.cc:2410] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:19:02.829540 20093 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.830370 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.800602 20098 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.831408 20098 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.832083 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.825559 20086 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.821987 20017 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.833897 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:02.835137 20086 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.837302 20086 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:02.837670 20188 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:19:02.838039 20088 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.838748 20088 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:02.835175 20036 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:02.840327 20017 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.842408 20017 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:02.844278 20086 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:02.843495 17409 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:02.844795 20086 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:02.845301 20086 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:02.843788 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:02.847656 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.848819 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:02.868561 19950 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:02.818819 20067 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:02.823210 20113 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:02.871590 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 430099us
W20260709 13:19:02.924988 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 132462us
I20260709 13:19:02.862957 20088 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:02.860426 20028 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:02.914105 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:02.947053 17410 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:02.867465 20026 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:02.791549 20185 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.949062 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:02.927896 19999 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:02.916313 20067 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:02.989418 17410 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:19:03.000749 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45251.5us
I20260709 13:19:03.002736 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.003283 19911 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:03.003945 20189 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:02.943168 19996 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:03.026635 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.041771 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.044348 16668 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:02.943058 20128 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:03.035763 16708 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.559s (SENT)
I20260709 13:19:03.047514 16662 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:03.049427 16663 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:19:02.995570 19560 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.700s	user 0.009s	sys 0.001s
I20260709 13:19:03.052618 16670 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:02.998425 20185 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.054126 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:03.056210 17414 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:03.057175 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.057963 16662 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:02.943099 19765 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:19:02.928277 20113 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:02.952060 20026 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:02.987936 20038 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.004526 20189 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.961743 20114 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.036625 19996 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:03.068490 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.088814 16670 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:02.983319 19830 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:03.073647 16662 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:03.077087 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:02.995468 19837 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:02.993076 20140 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.068388 20098 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.086424 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.100056 17412 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:19:03.065531 20186 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.342s	user 0.002s	sys 0.000s
I20260709 13:19:03.078991 20026 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.103192 16662 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:03.074903 19765 consensus_queue.cc:1243] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:19:03.102205 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 153237us
I20260709 13:19:03.086140 20189 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:03.102150 16636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.016s	sys 0.020s
I20260709 13:19:03.115965 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.112356 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.113462 17374 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:03.116746 16636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.016s	sys 0.020s
I20260709 13:19:03.100723 20035 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:03.098184 19830 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:03.097260 20191 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.067590 19560 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.132074 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:03.133137 20018 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:03.134056 20018 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:03.134549 20018 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.134976 20018 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:03.137136 19996 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.138191 19996 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:03.143594 20182 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.528s	user 0.009s	sys 0.000s
I20260709 13:19:03.144062 20182 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.144685 20036 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:03.146448 20182 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.147001 20035 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:03.148022 20035 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.148376 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.149271 16662 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:03.150738 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:03.152139 20038 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:19:03.152714 20038 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:03.153612 20020 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:03.155154 16666 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:03.156491 16595 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:03.152475 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.159690 20080 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.160348 20080 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:03.160509 20113 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.160897 20080 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:03.156224 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.159834 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.167981 16662 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:03.173663 20113 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:03.117633 20021 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:03.166081 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.177795 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:03.178836 16595 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
W20260709 13:19:03.117316 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 667033us
W20260709 13:19:03.113732 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 251892us
I20260709 13:19:03.097443 20148 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:03.100194 19837 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.124922 20087 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:19:03.126773 20192 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:19:03.127130 19577 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:03.216274 20021 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:03.127600 19830 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.171419 20020 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:03.221647 20080 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.131049 20191 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.207981 20067 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.211071 20148 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:03.214953 20087 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:03.218575 16587 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:19:03.219313 19830 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:03.210623 20086 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.177232 20099 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.226877 20026 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.228420 20189 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.234520 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:03.235136 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51749.6us
I20260709 13:19:03.254351 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:03.235266 20038 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:03.246666 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 80675.2us
I20260709 13:19:03.256757 16737 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:03.253242 16670 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:03.248136 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.251639 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.257557 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.246264 20128 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:03.261960 16664 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:03.261690 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.259342 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:03.202760 20018 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.268795 20189 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:03.222906 20021 consensus_queue.cc:1175] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "170921ba08b74a33995ae7c7686f8109" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:03.271401 20125 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:03.274773 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.275404 20204 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:03.301427 20204 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:19:03.301956 20202 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.143s	user 0.003s	sys 0.000s
I20260709 13:19:03.276276 20020 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:03.277870 20099 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:03.286253 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.308033 20185 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:03.288609 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.324872 16742 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:03.295846 19577 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:03.296547 20148 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.298748 20018 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.300034 19758 raft_consensus.cc:2764] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:19:03.290186 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.260382 20127 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:03.249869 20017 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:19:03.253750 20113 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:19:03.256251 20205 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:19:03.336818 20003 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:03.367904 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 248988us
I20260709 13:19:03.369220 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:03.341202 20185 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:03.377347 19577 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:03.377465 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:03.380813 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.360226 20191 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.390455 16742 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:03.395037 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:03.395973 17409 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:03.397550 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.399245 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.399999 17412 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:03.410441 17409 raft_consensus.cc:2368] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:19:03.412120 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:03.413384 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 123297us
I20260709 13:19:03.414438 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.422081 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.336833 20087 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.388331 19577 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.393316 20128 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:03.429118 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.374900 19837 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.375511 20003 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:03.441949 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 259469us
W20260709 13:19:03.439670 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 256217us
I20260709 13:19:03.429419 20020 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:03.429965 20191 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:03.454550 16561 leader_election.cc:400] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:19:03.455653 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:03.456025 16561 leader_election.cc:403] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:03.459252 16740 raft_consensus.cc:2368] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:03.440254 20127 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:03.441309 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:03.443913 20017 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.467617 19765 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:03.491230 20017 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.494050 20017 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.468538 19950 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:03.466534 20026 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.495059 19950 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.475605 20035 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:03.480048 20003 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.480108 19837 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.481915 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:03.498885 17420 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.498762 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:03.451292 20087 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.464056 20140 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.462690 20086 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.490549 19560 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.495494 19950 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:03.499430 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:03.506096 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 231382us
I20260709 13:19:03.506990 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:03.514989 17420 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:03.501708 20148 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:03.508929 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 215763us
I20260709 13:19:03.503149 20127 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.530584 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:03.505048 20035 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:03.536791 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.524720 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:03.467188 20185 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.504065 20118 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:19:03.507411 20140 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.552280 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:03.552584 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:03.527130 20086 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:03.549257 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.510924 20125 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:03.555791 16594 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:03.556825 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.556964 17420 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:03.528986 20020 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:19:03.558353 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.558751 16587 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:03.559108 16743 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:03.559418 16742 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:19:03.559742 16742 pending_rounds.cc:85] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 5
I20260709 13:19:03.559953 16742 pending_rounds.cc:107] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:19:03.560210 16742 raft_consensus.cc:2889] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.521450 20206 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:19:03.565490 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.519194 19830 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:19:03.539551 20067 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:03.567786 17413 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:03.521019 19837 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:19:03.570375 17416 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:03.557144 20086 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.575278 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.575170 20067 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.576155 20140 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.576843 16742 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:03.577627 20140 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:03.578076 20140 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.578414 20140 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:03.580200 20206 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:03.580634 20206 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:03.580986 20206 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:03.583200 16737 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:03.584980 19765 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:19:03.575289 20086 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:03.547183 20127 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.592798 20114 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:03.593214 20114 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:03.593782 20127 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.593974 20114 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.594611 20035 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.627425 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.629138 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.528430 20148 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.631733 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.633949 16743 raft_consensus.cc:2410] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:19:03.644793 16742 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:19:03.645262 16742 pending_rounds.cc:85] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:19:03.645501 16742 pending_rounds.cc:107] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:19:03.645746 16742 raft_consensus.cc:2889] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.647282 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.648653 16742 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:03.650714 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.651665 16741 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:19:03.652266 17376 leader_election.cc:400] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:03.652760 17376 leader_election.cc:403] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:03.657670 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.658614 17420 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:03.661612 16740 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:19:03.662083 16740 pending_rounds.cc:85] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 5
I20260709 13:19:03.662317 16740 pending_rounds.cc:107] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:19:03.662550 16740 raft_consensus.cc:2889] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:03.670583 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.671281 16743 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:19:03.672058 16739 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:03.674909 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:03.643303 20126 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.684511 16745 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:03.656533 20127 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.669009 19577 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:03.719127 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:03.721158 16739 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:19:03.733296 20219 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:19:03.755067 16636 connection.cc:256] RPC call timeout handler was delayed by -1.8189577439998175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
W20260709 13:19:03.758838 16636 connection.cc:256] RPC call timeout handler was delayed by -1.7622577439999532s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260709 13:19:03.761528 16636 connection.cc:256] RPC call timeout handler was delayed by -1.7064577439998629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
W20260709 13:19:03.764075 16636 connection.cc:256] RPC call timeout handler was delayed by -1.6695577439998033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 13:19:03.764418 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:03.767443 16636 connection.cc:256] RPC call timeout handler was delayed by -1.3869577439998011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22599999999999998s for a response
W20260709 13:19:03.769452 16636 connection.cc:256] RPC call timeout handler was delayed by -1.1547577439998804s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25179999999999997s for a response
I20260709 13:19:03.716410 20099 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:03.717018 20125 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:03.710276 20126 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:03.787736 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 262927us
I20260709 13:19:03.792555 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:03.793183 20126 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.793375 16670 raft_consensus.cc:606] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:03.793864 20126 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:03.793949 16670 raft_consensus.cc:614] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:03.729103 20093 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:03.799345 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.803943 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.776403 19560 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.720072 20127 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.782269 20099 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:03.841531 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45753.3us
I20260709 13:19:03.794637 20126 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:03.890353 19810 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.847169 19560 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:03.853257 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 243791us
I20260709 13:19:03.894407 19810 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:03.895440 20148 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:03.896020 19560 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:03.896620 20236 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:19:03.896826 19810 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:03.902603 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.903316 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:03.891439 20232 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 13:19:03.903354 17420 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:03.903373 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:03.907297 16665 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:03.910116 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:03.910840 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.912051 17420 raft_consensus.cc:2410] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:19:03.859458 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.856101 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.917416 16664 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:03.914673 16743 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:03.849709 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:03.907761 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97416.8us
I20260709 13:19:03.932245 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:03.813982 20035 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:03.964816 17412 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:03.975277 16665 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
W20260709 13:19:03.955727 20228 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:19:03.807300 20093 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.011698 20093 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:03.982333 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:04.015839 20229 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:19:04.017071 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.018898 20126 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.023589 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.032599 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.034324 16665 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:04.039294 20224 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:04.039466 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57241us
I20260709 13:19:04.040771 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:04.042140 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:04.042953 20223 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.043426 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.043581 20223 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.044129 20224 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:04.044402 20242 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.044883 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:03.814644 19950 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.031363 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:04.046542 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 516051us
W20260709 13:19:04.032040 20238 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:19:04.048767 16664 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:04.050827 16636 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:04.051303 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:03.966715 19577 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:03.979717 20162 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:04.048270 20093 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:04.079556 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:04.080168 20093 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:04.080554 20088 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:04.080914 20088 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:04.081164 20093 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.081707 20088 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.082464 20148 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:04.083151 20148 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:04.071801 20224 raft_consensus.cc:1064] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:19:04.073030 20242 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:04.075281 16744 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:03.951164 20239 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:19:04.077214 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:03.978032 20035 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.062767 20235 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.083604 20086 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.053341 20240 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:19:04.085093 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:04.087374 20140 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.508s	user 0.011s	sys 0.000s
I20260709 13:19:04.068626 20223 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:04.049880 20225 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:04.105130 20088 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:04.089048 17374 leader_election.cc:336] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.791s (SENT)
I20260709 13:19:03.983522 20099 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.116817 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.061529 19577 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.079460 20245 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:04.100195 16494 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 6, term changed from 6 to 8, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:04.108904 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.113417 16662 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 7: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:04.083204 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:04.118350 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97085.5us
I20260709 13:19:04.128003 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.128908 16665 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:19:03.978771 20125 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.131415 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.075657 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:04.133237 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.133864 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.135075 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.136466 16662 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:04.138271 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.141815 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.142472 16663 raft_consensus.cc:2368] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:04.142583 20035 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.143514 19950 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.111591 20235 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.144102 20067 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.144735 20067 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.145743 20235 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.146423 19577 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.147879 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1395) took 1123 ms. Trace:
I20260709 13:19:04.148245 16666 rpcz_store.cc:276] 0709 13:19:03.024580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.026184 (+  1604us) service_pool.cc:224] Handling call
0709 13:19:03.051430 (+ 25246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.155570 (+104140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.147648 (+992078us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540008b454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:04.147819 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238055680}
I20260709 13:19:04.133299 20099 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:04.152221 20214 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.441s	user 0.005s	sys 0.007s
I20260709 13:19:04.092754 20162 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.156584 20162 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:04.159139 20114 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:04.136677 20206 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.002s
I20260709 13:19:04.167862 20206 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.170040 20206 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.172981 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 2002) took 2297 ms. Trace:
I20260709 13:19:04.173231 20086 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.116273 20140 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.175400 20140 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:04.131956 20126 consensus_queue.cc:1175] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 7 status { last_received { term: 5 index: 6 } last_committed_idx: 6 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:04.140686 20148 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.173404 16667 rpcz_store.cc:276] 0709 13:19:01.876044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:01.876757 (+   713us) service_pool.cc:224] Handling call
0709 13:19:04.172755 (+2295998us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540033ac54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:04.172898 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3114139392}
I20260709 13:19:04.139626 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:04.193591 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 112481us
I20260709 13:19:04.200858 16710 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:04.224138 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.230608 16662 raft_consensus.cc:1217] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 7.7->[9.8-9.8]   Dedup: 9.8->[]
I20260709 13:19:04.231232 16662 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:19:04.240590 20161 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:04.241106 20161 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:04.242007 20161 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.244782 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.245978 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.247303 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.247932 16666 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 6.
I20260709 13:19:04.248826 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:04.249267 16665 raft_consensus.cc:2410] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:19:04.249384 16670 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:04.251598 16494 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:04.255332 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:04.256261 16664 raft_consensus.cc:2368] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:04.257221 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.257745 16670 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:04.260666 16739 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:19:04.263324 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.264199 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:04.268395 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:04.268798 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 179789us
I20260709 13:19:04.269236 16742 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
W20260709 13:19:04.271329 17374 leader_election.cc:336] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.202s (SENT)
I20260709 13:19:04.271806 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.272143 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:04.274331 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:04.275272 17420 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:04.274446 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:04.230401 20128 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:04.279819 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:04.253865 16494 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:04.280666 16739 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:19:04.238770 16493 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 140) took 25782 ms (client timeout 15000 ms). Trace:
I20260709 13:19:04.283066 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:04.283926 16741 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:04.284400 17412 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:19:04.287585 17374 leader_election.cc:400] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:04.286933 16494 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 5, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 5 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:04.290104 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:04.290732 17410 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:19:04.293288 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.295066 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:04.277311 19577 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:04.273605 20140 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.285636 20128 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.300292 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:04.300318 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:04.301669 20099 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:19:04.291373 16494 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 8, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), now has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:04.299875 20184 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:04.307694 20184 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.307566 20252 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:04.308184 20184 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:04.307972 16494 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:04.309338 16494 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 5, term changed from 4 to 6, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to <none>, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:04.286391 20235 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.305925 20240 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:19:04.311208 20235 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:04.305508 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 170205us
I20260709 13:19:04.313301 16494 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 4, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 4 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:04.317931 20235 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.323371 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:04.313241 20140 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.324122 20184 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.325937 20140 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.327082 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.327857 20225 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:04.328647 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:04.329423 20140 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:04.329804 20140 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.330174 20140 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:04.293535 19758 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.332682 19758 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.300442 20161 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:04.313984 20128 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:04.345163 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.353118 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.337440 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:04.284552 16493 rpcz_store.cc:269] 0709 13:18:38.456625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:38.456888 (+   263us) service_pool.cc:224] Handling call
0709 13:19:04.238654 (+25781766us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:18:59.341228 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:18:59.345197 (+  3969us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:18:59.345255 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:18:59.345272 (+    17us) tablet.cc:662] Decoding operations
0709 13:18:59.480924 (+135652us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:18:59.512287 (+ 31363us) write_op.cc:641] Partition lock acquired for write op
0709 13:18:59.515200 (+  2913us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:18:59.705917 (+190717us) tablet.cc:701] Row locks acquired
0709 13:18:59.728919 (+ 23002us) write_op.cc:260] PREPARE: finished
0709 13:18:59.766981 (+ 38062us) write_op.cc:270] Start()
0709 13:18:59.795245 (+ 28264us) write_op.cc:276] Timestamp: P: 1783603139760193 usec, L: 0
0709 13:18:59.797224 (+  1979us) op_driver.cc:348] REPLICATION: starting
0709 13:18:59.900425 (+103201us) log.cc:844] Serialized 3170 byte log entry
0709 13:19:01.124593 (+1224168us) op_driver.cc:464] REPLICATION: finished
0709 13:19:01.852461 (+727868us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327cada0ac 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f328361aef0
0709 13:19:01.948134 (+ 95673us) write_op.cc:301] APPLY: starting
0709 13:19:01.955617 (+  7483us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:01.969875 (+ 14258us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:01.969921 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:01.986135 (+ 16214us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:01.986453 (+   318us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:19:01.986498 (+    45us) write_op.cc:312] APPLY: finished
0709 13:19:01.986851 (+   353us) log.cc:844] Serialized 65 byte log entry
0709 13:19:01.987071 (+   220us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:01.987198 (+   127us) write_op.cc:454] Released schema lock
0709 13:19:01.987275 (+    77us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:01.987774 (+   499us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:01.990487 (+  2713us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2988429,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":814254,"apply.run_cpu_time_us":3599,"apply.run_wall_time_us":48543,"num_ops":8,"prepare.queue_time_us":89770,"prepare.run_cpu_time_us":39523,"prepare.run_wall_time_us":1045696,"raft.queue_time_us":205789,"raft.run_cpu_time_us":26267,"raft.run_wall_time_us":821248,"replication_time_us":1323833,"spinlock_wait_cycles":47753472,"thread_start_us":1474894,"threads_started":4,"wal-append.queue_time_us":646729,"wal-append.run_cpu_time_us":9876,"wal-append.run_wall_time_us":2427850}]]}
I20260709 13:19:04.409422 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:04.410322 16668 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.411441 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.413801 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.421298 16665 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:04.425848 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.426203 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.427160 16664 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:04.351171 20240 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:04.444309 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.346577 20127 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:04.454048 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.414718 16662 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:04.356001 19577 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.468418 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.470989 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.472126 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.472970 16670 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:04.399097 20018 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:04.400257 19758 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.475981 19577 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.480469 16668 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:04.484835 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.461351 20204 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.485997 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:04.476158 20018 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.496270 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:04.476325 19549 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:04.492355 20223 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:04.483182 20184 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.489555 20244 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:04.499745 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:04.486960 20240 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:04.491770 20162 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:04.476207 19560 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 13:19:04.499529 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 202107us
W20260709 13:19:04.519759 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.500108 20018 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.480690 20225 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:04.501394 20242 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:04.492754 20125 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:04.490988 20239 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:19:04.563771 16592 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:04.563612 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 896353us
I20260709 13:19:04.565168 16596 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:04.568382 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.575793 16594 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:04.500491 20204 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:19:04.578616 16595 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:04.594949 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.595923 16596 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:04.602710 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 551513us
I20260709 13:19:04.575062 20223 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:04.572458 20242 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.614156 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.579691 20162 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:04.622500 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:04.629771 16596 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:04.611681 19999 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.610538 20112 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:04.640774 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 141080us
I20260709 13:19:04.619321 20223 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.620098 20242 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:04.676249 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.635413 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:04.685667 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 138001us
I20260709 13:19:04.632529 20206 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:19:04.743942 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 139868us
I20260709 13:19:04.795789 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.797006 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.798696 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:04.799582 16593 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:04.787848 16595 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:04.699191 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:04.802114 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 103045us
W20260709 13:19:04.769657 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 75226.6us
I20260709 13:19:04.693667 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:04.802917 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.816921 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.817705 16588 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:04.684368 20223 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:04.818845 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:04.820048 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 299610us
W20260709 13:19:04.821187 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.821774 16589 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:19:04.822263 16589 pending_rounds.cc:85] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 6
I20260709 13:19:04.822533 16589 pending_rounds.cc:107] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:19:04.822834 16589 raft_consensus.cc:2889] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:04.824011 20276 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:04.824463 20276 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:04.824802 20276 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:04.689960 20112 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.834628 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2740) took 1275 ms. Trace:
I20260709 13:19:04.853226 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.855749 16595 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.801383 20206 consensus_queue.cc:1243] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:04.756503 19577 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.745808 20239 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.809928 20217 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.859385 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
W20260709 13:19:04.690913 20266 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:19:04.867306 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.628600 20205 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.633975 20246 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:04.868788 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:04.828310 20204 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.692668 19549 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.886435 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:04.887645 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.888688 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.889894 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:04.871599 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49779.6us
I20260709 13:19:04.892781 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:04.899003 16665 raft_consensus.cc:2410] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:19:04.634145 19950 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.236s
I20260709 13:19:04.802389 20140 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.869813 20185 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:04.800746 20162 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.868144 20003 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:19:04.891803 20112 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:04.924489 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.924948 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.926101 16588 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:04.908989 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:04.688027 20240 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.912166 20218 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:04.923995 20161 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.002s
I20260709 13:19:04.894980 20217 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.900471 19549 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.880940 20239 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.937722 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:04.940809 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:04.936820 20258 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:19:04.917091 20204 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:19:04.894258 20205 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:04.959233 20261 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:19:04.959316 20204 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:04.959892 20204 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:04.960789 20205 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.951066 20161 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:04.961585 20161 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:04.962024 20161 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:04.951350 20274 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:19:04.960834 20204 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.964046 20204 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:04.965039 20140 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:04.939122 20104 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:19:04.952719 20018 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:04.968669 20018 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:04.968756 20239 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.972292 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:04.974103 19549 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:04.957562 16595 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:04.940590 20218 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.976886 20218 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:04.976960 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:04.977790 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 177783us
I20260709 13:19:04.978268 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:04.977627 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 301440us
I20260709 13:19:04.897832 20246 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.984339 16561 leader_election.cc:400] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7.
I20260709 13:19:04.927083 20275 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:19:04.972324 20128 consensus_meta.cc:287] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.001s
I20260709 13:19:05.003005 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.003724 16665 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:05.004993 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.004647 16590 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:04.999408 20293 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:04.977525 20279 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:19:05.014887 16594 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:04.991123 19950 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:04.991400 20067 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:04.995961 20003 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:19:05.005951 16595 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:19:05.006574 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.024899 16588 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:04.966642 20223 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.003167 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:04.985461 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.007927 20284 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:19:05.016438 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:04.986159 20218 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:05.029163 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.029457 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.030352 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.033468 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:05.035887 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.039119 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:04.978886 20161 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.043537 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.044273 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2639) took 1442 ms. Trace:
I20260709 13:19:05.020701 19950 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:05.015348 20253 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.046433 16737 rpcz_store.cc:276] 0709 13:19:03.601610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.601845 (+   235us) service_pool.cc:224] Handling call
0709 13:19:03.602224 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:03.633934 (+ 31710us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:04.349779 (+715845us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.516097 (+166318us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.531704 (+ 15607us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.531891 (+   187us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:04.531937 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.532045 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:05.044070 (+512025us) raft_consensus.cc:1685] finished
0709 13:19:05.044091 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.044210 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":724433,"prepare.queue_time_us":721101,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":75209,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":27551,"thread_start_us":165859,"threads_started":1,"wal-append.queue_time_us":267907}
I20260709 13:19:04.957617 20276 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.051463 20224 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:04.835347 16587 rpcz_store.cc:276] 0709 13:19:03.559553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.570552 (+ 10999us) service_pool.cc:224] Handling call
0709 13:19:03.639444 (+ 68892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.639673 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:19:03.656226 (+ 16553us) log.cc:844] Serialized 14 byte log entry
0709 13:19:03.659852 (+  3626us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.518125 (+858273us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b1800361ea0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:19:04.745090 (+226965us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.817100 (+ 72010us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:04.817151 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:04.817197 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.817260 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:04.834538 (+ 17278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":940494,"prepare.queue_time_us":81446,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":5983,"raft.queue_time_us":968431,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":13505,"spinlock_wait_cycles":20162048,"thread_start_us":225716,"threads_started":1}
W20260709 13:19:05.053748 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.054306 20294 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:05.034847 19999 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:04.996963 19577 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:19:05.061482 16590 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:05.021257 20293 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.019331 20128 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.040915 19549 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 13:19:05.089126 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.100941 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.068166 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.107457 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.114292 16666 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:05.119958 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.122053 16590 raft_consensus.cc:2368] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8.
W20260709 13:19:05.122150 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 119113us
I20260709 13:19:05.125664 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:05.126958 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.128165 20213 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:05.134045 20213 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:05.134428 20302 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.135592 20213 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.059743 19950 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:05.066232 20099 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:05.140770 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.081075 20217 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.061164 20240 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:05.161888 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.126977 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.070210 19577 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:05.170331 16587 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:05.177271 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.141438 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.066344 20162 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 13:19:05.081252 20244 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:05.072144 20246 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.175550 16590 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
W20260709 13:19:05.170837 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.180595 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.067845 20253 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.167362 20166 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:05.172816 20240 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:05.184020 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.187350 16589 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:05.190918 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.191072 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:05.188490 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.194572 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.195474 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.169902 20099 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:05.169245 20302 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:05.161128 20242 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.089004 20294 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.213567 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:05.188782 19577 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:05.213578 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:05.189894 20166 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:05.219513 16665 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:05.212602 16589 raft_consensus.cc:2368] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:05.220386 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.198875 20099 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.223543 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.199925 20275 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.199088 20223 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.193207 20299 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:05.242686 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 80831.9us
I20260709 13:19:05.243012 16668 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.243603 16587 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.251657 16668 raft_consensus.cc:614] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:05.193795 20204 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:05.253221 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:05.253505 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:05.253894 17420 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:05.200397 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:05.254050 17415 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:05.254014 16589 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:05.189548 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.247172 16663 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:05.255374 16590 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:05.204393 20302 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.256623 16590 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:19:05.256734 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.261509 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:05.261951 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.262108 16587 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:05.192764 20244 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.263991 16587 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:05.263418 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:05.270471 17420 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:05.271947 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.272636 16663 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:05.273631 16663 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:05.274744 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:05.277578 16665 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:05.277480 17374 leader_election.cc:400] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:05.278460 17415 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:05.279326 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.279639 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:05.252311 20276 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.288358 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:05.274104 16590 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:05.286299 20303 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:05.249689 20240 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.291323 20112 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:05.292984 20223 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:05.293908 20112 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:05.271631 20217 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:05.294736 20217 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.286577 20234 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.297626 20234 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.298151 20234 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.294973 20112 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.291711 20218 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:19:05.272388 20166 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.302258 20218 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:05.302793 20218 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:05.304600 20218 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.306253 20218 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:05.323359 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:05.324790 16595 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:05.275113 20093 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:05.264447 20205 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:19:05.302623 20166 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.258417 20204 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.334901 16665 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:05.339617 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:05.251852 20275 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.284777 20161 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.367916 16589 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:05.375967 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.553s (SENT)
I20260709 13:19:05.376858 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:05.369326 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:05.379956 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.381230 20003 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:05.381924 20003 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:05.383144 20003 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.399801 20302 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.335088 20128 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.387298 19577 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:05.334369 20304 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:05.372152 20246 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:19:05.401408 20281 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.442s	user 0.005s	sys 0.000s
I20260709 13:19:05.390275 20293 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.257121 20299 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.413450 20275 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.413591 20204 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.368894 20303 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:05.372856 20093 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:05.442651 20287 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.386s	user 0.000s	sys 0.003s
I20260709 13:19:05.461751 20240 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:05.422840 19577 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:05.468809 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92894.4us
I20260709 13:19:05.436311 20246 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:05.457616 20293 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:05.482537 16665 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:05.470522 20253 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.471609 19577 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:05.473156 20244 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.475689 20242 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.488422 20294 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.497534 20234 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.518780 20299 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.518491 19950 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:05.521281 20264 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.615s	user 0.002s	sys 0.000s
I20260709 13:19:05.521633 19950 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.514232 20218 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.539189 20218 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.487517 20246 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.540676 20246 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.544664 20218 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.545514 20217 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.493791 20093 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:05.554109 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.538240 20099 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.566640 20099 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.530390 20234 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.585067 16670 raft_consensus.cc:2410] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:19:05.587141 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 572) took 1301 ms. Trace:
I20260709 13:19:05.587586 17412 rpcz_store.cc:276] 0709 13:19:04.285836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.287249 (+  1413us) service_pool.cc:224] Handling call
0709 13:19:04.287549 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:04.287889 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:04.287913 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:04.288361 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 13:19:04.502661 (+214300us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:04.515324 (+ 12663us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.515631 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:04.515735 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:04.976189 (+460454us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b180037a860. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f3283ab858f 00007f327caf7571 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd
0709 13:19:05.579345 (+603156us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b180037a860. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1
0709 13:19:05.580914 (+  1569us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b180037a860. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:05.583356 (+  2442us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b180037a860. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:05.586880 (+  3524us) raft_consensus.cc:1685] finished
0709 13:19:05.586907 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.587068 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2722,"prepare.queue_time_us":13783,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":1978,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":32513,"spinlock_wait_cycles":135714176,"thread_start_us":187532,"threads_started":1,"wal-append.queue_time_us":246189}
I20260709 13:19:05.596266 20302 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.601995 16664 raft_consensus.cc:2410] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:19:05.610857 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:05.611377 17374 leader_election.cc:400] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:05.611936 17374 leader_election.cc:403] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:05.615631 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.616307 16668 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:05.617642 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:05.617684 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.614951 17409 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:05.611574 17412 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:05.618988 20302 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.622346 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:05.625445 20280 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:05.627204 20280 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:05.627403 20166 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:05.627805 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:05.628177 20280 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.628773 20189 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:05.629207 20189 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.629565 20189 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.630692 20280 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:05.624733 20003 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:05.624881 19577 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.636752 20184 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:05.638223 20189 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.641906 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:05.652807 16666 raft_consensus.cc:606] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.651177 20003 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.653301 16666 raft_consensus.cc:614] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:05.621646 20224 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:19:05.653959 20003 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.652431 20184 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:05.664809 19577 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:05.667261 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.665344 20267 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.985s	user 0.002s	sys 0.003s
I20260709 13:19:05.672720 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:05.677804 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:05.678509 16670 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.678957 16670 raft_consensus.cc:614] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:05.680379 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2636) took 2087 ms. Trace:
I20260709 13:19:05.680770 16738 rpcz_store.cc:276] 0709 13:19:03.592671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:03.634670 (+ 41999us) service_pool.cc:224] Handling call
0709 13:19:03.656207 (+ 21537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:03.656432 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:04.400865 (+744433us) log.cc:844] Serialized 14 byte log entry
0709 13:19:04.631427 (+230562us) log.cc:844] Serialized 14 byte log entry
0709 13:19:05.330000 (+698573us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b1800359ec0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:05.679946 (+349946us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b1800359ec0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:19:05.680085 (+   139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:05.680128 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:05.680174 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.680231 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.680328 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":708065,"prepare.queue_time_us":827231,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":10019,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":24377,"spinlock_wait_cycles":33528832,"thread_start_us":162789,"threads_started":1,"wal-append.queue_time_us":539735}
I20260709 13:19:05.681854 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:05.682615 16670 raft_consensus.cc:606] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.683794 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:05.689666 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:05.690299 16666 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:05.691141 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.692608 20038 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:05.693585 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.693827 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.695115 20269 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:05.695148 20018 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:05.695691 20269 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.696364 20018 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.697122 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2648) took 1438 ms. Trace:
W20260709 13:19:05.697777 16708 connection.cc:256] RPC call timeout handler was delayed by -1.2234660110000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 13:19:05.697526 16743 rpcz_store.cc:276] 0709 13:19:04.258898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.259811 (+   913us) service_pool.cc:224] Handling call
0709 13:19:04.279027 (+ 19216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:04.279292 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:05.082610 (+803318us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b1800483700. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327f78c4bc 00007f327cada48b 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf4455 00007f32835dc3c7 00007f327fbee9a7 00007f327fbfe0bf
0709 13:19:05.082814 (+   204us) log.cc:844] Serialized 14 byte log entry
0709 13:19:05.696795 (+613981us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:05.696848 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:05.696891 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:05.696958 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:05.697071 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":708983,"prepare.queue_time_us":291650,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"spinlock_wait_cycles":9978112,"thread_start_us":231907,"threads_started":2,"wal-append.queue_time_us":140087}
I20260709 13:19:05.700347 16666 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:05.701730 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:05.667367 20189 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.709630 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.710353 20067 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:05.710973 20067 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:05.711488 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.711489 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.712194 20067 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.712670 20184 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.713822 20204 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:05.714129 20184 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:05.714411 20204 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.716379 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:05.716724 20204 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.716930 20018 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.718197 20233 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.718600 20269 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.718721 20233 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.719236 20209 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:05.723918 17415 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:19:05.723765 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:05.730113 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.731011 16743 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:05.699618 17409 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:05.693109 20038 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.736023 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:05.736702 17413 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:19:05.737610 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.735360 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.738493 16743 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:05.739360 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.739492 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.757608 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.764460 16740 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:05.764550 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.756811 16738 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:05.766705 16739 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:05.766937 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.767899 16595 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:05.768196 17412 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:05.721412 20233 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.762545 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.732854 20018 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.739447 16741 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:05.770900 16742 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:05.771055 16739 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:19:05.732113 19758 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:05.768829 16738 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:05.777817 17420 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:05.778437 17413 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
W20260709 13:19:05.782562 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:05.783429 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44853.7us
I20260709 13:19:05.783557 16744 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:05.771306 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 72929.2us
I20260709 13:19:05.785547 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.786173 16740 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:05.788025 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.789992 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.790618 16595 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:05.791006 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.791579 16745 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:05.792106 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.792832 16595 raft_consensus.cc:2368] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11.
I20260709 13:19:05.793502 19577 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:05.794049 19577 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.795245 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.798154 16740 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:05.801903 20209 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:05.802543 20209 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.789486 20018 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.812306 16745 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:05.813580 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.815493 16740 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.818218 16742 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:05.822782 16738 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 13:19:05.790233 20233 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.826542 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.828043 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.827318 16664 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:05.830025 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:05.831749 20209 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.833060 20302 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:05.789274 19758 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.836198 19758 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.840096 20038 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.840418 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:05.851022 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:05.814891 20148 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:05.864678 20148 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.865458 20260 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:05.867040 20254 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:05.867540 20118 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:19:05.867962 20254 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:05.868320 20254 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:05.868383 20118 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:05.868816 20118 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:05.869222 20254 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.869611 20118 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.858723 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:05.860550 19911 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:05.861928 20018 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.863746 20003 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.872229 16710 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:05.880611 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:05.870492 20254 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.894994 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.895959 16740 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:05.897294 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:05.892935 20003 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.888913 20017 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:19:05.898192 20038 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:05.890105 17416 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:05.892252 16741 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:19:05.902967 20035 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:05.903407 20035 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:05.903751 20035 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:05.904834 20018 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:05.905351 20018 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:05.906183 20018 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:05.911633 20191 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:05.912130 20191 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:05.912503 20191 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:05.914637 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:05.892786 20148 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:05.920459 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:05.931646 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.889482 20118 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.915515 20304 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:05.954067 20304 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:05.954447 17415 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:19:05.847232 20233 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:05.898784 20038 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:05.961187 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:05.961849 17420 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.964152 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.964819 16592 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:05.941869 19577 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:05.970091 17414 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:05.973919 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:05.990231 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.990790 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.991056 16596 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:05.991400 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.992020 16593 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:05.991072 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:05.994078 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:05.991046 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:05.994935 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:05.995326 16588 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:05.995525 16595 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:05.997982 16744 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:05.954509 20304 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:05.998461 20112 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:05.998191 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:05.978446 20284 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:06.007417 20284 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.009740 20284 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:05.998713 20234 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:06.011355 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:05.994251 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:19:06.012975 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.014133 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.016304 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.016367 16595 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:06.014319 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:06.019966 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.020488 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:06.022328 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:06.025019 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.025640 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.027240 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.028350 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.028661 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.029086 16589 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:06.029381 16591 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:06.029635 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:06.030428 16594 raft_consensus.cc:2410] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:19:06.030507 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.031356 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:06.031941 16669 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:06.032536 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:06.035214 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:06.038516 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.039253 16591 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:06.040772 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.041349 16591 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:06.043398 20038 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:06.047824 20217 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:06.048533 20217 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:06.049672 20217 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.053409 16668 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:06.055787 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:06.056663 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 134945us
W20260709 13:19:06.058118 16708 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.988s (SENT)
I20260709 13:19:06.060683 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.020781 16744 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:06.067873 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.071803 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:06.074641 16708 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.057s (SENT)
I20260709 13:19:06.010417 20234 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:06.073410 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.077947 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:06.078153 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:06.078826 17419 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:06.079371 16744 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:06.072166 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:06.084051 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.084600 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.090055 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.090962 16665 raft_consensus.cc:2410] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:19:06.095758 16740 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:19:06.101496 16740 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:06.089277 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:06.103847 17420 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:06.075232 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.023448 20280 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:06.106503 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:06.108500 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:06.189620 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 81194.9us
I20260709 13:19:06.193068 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:06.194392 16594 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:19:06.195811 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:06.195977 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:06.135409 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.163839 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:06.197009 16738 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:06.086437 20234 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:06.201730 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.017253 20246 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 13:19:06.195019 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:06.202575 16667 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
W20260709 13:19:06.184536 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109915us
I20260709 13:19:06.199285 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:06.197618 16662 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
W20260709 13:19:06.211117 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.212121 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.099900 20067 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:06.213029 16665 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:06.220847 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.221506 16738 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:19:06.225283 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.226436 16737 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:06.233039 16592 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:06.240070 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:06.123538 20113 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:06.240072 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.220494 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.099565 19577 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 13:19:06.112665 20312 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:06.246650 16667 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:19:06.248358 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.623s (SENT)
I20260709 13:19:06.203262 20269 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:06.219877 20036 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:06.268219 20036 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:06.268754 20036 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:06.271577 20333 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:19:06.276840 20233 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:06.281896 20191 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:06.284693 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.208902 20028 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:19:06.288188 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:06.288900 20113 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.128701 20148 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.123s
W20260709 13:19:06.293742 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.220705 20087 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:06.297623 20087 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:06.298152 20087 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:06.299475 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.301510 16591 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:06.128636 20302 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:19:06.305255 20302 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:06.305980 20302 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.221562 20003 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:06.308250 20003 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:06.308471 20113 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:06.309176 20003 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:06.319651 20184 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.011s
I20260709 13:19:06.323066 20067 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:06.325205 19577 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:06.326026 19577 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.329120 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:06.329927 16663 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:06.343339 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.289019 16737 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:06.355552 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:06.360275 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:06.361220 16745 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:06.361654 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.366539 16663 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
W20260709 13:19:06.373683 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 125355us
I20260709 13:19:06.249809 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:06.260077 16594 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:06.203401 20218 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:06.386045 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:06.395920 16745 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:06.384944 16739 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:19:06.237047 16595 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:06.400668 16595 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:06.403767 16663 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:06.403308 16634 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:06.421303 16737 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
W20260709 13:19:06.449350 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.452284 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.511109 16592 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:06.512228 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.552798 16667 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:06.555234 16745 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:19:06.567889 19577 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:06.570188 19577 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:06.570441 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:06.573531 19577 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.580600 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:06.569078 16669 consensus_meta.cc:287] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.537s	user 0.012s	sys 0.003s
W20260709 13:19:06.589262 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.590123 16669 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
W20260709 13:19:06.479369 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.592301 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 140085us
W20260709 13:19:06.593362 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.594569 16663 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [5.4-8.6]
I20260709 13:19:06.404985 20148 consensus_queue.cc:1243] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:06.364077 20304 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.624047 16738 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:06.404423 20233 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:06.407282 20225 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:06.593390 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.457554 20240 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:06.655735 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.637776 19577 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:06.638459 16670 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.003s
W20260709 13:19:06.660797 16670 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.011s	sys 0.003s
W20260709 13:19:06.639925 16668 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.006s	sys 0.008s
I20260709 13:19:06.383236 20125 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.198s
W20260709 13:19:06.584980 20205 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.042s	sys 0.056s
I20260709 13:19:06.646209 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.648262 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:06.663756 16668 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.007s	sys 0.008s
W20260709 13:19:06.669255 16664 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.003s	sys 0.012s
I20260709 13:19:06.674180 16669 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: [9.9-10.10]
W20260709 13:19:06.680299 16664 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.004s	sys 0.012s
I20260709 13:19:06.680603 20269 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:19:06.680871 16671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.001s	sys 0.015s
W20260709 13:19:06.681272 16671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.001s	sys 0.015s
W20260709 13:19:06.684329 17410 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.003s	sys 0.018s
W20260709 13:19:06.684806 17410 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.003s	sys 0.018s
I20260709 13:19:06.621466 19911 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:06.676990 16738 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:06.691286 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:06.695484 16745 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:19:06.663785 20035 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.760s	user 0.014s	sys 0.008s
I20260709 13:19:06.697171 20035 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:06.699658 16744 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:06.410406 20218 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:06.705466 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:06.685487 20205 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.042s	sys 0.057s
W20260709 13:19:06.699955 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 105790us
I20260709 13:19:06.734982 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.736913 16708 leader_election.cc:400] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:06.750092 16740 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:06.750720 16740 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:06.751744 16740 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:06.753278 16740 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:06.706993 20018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.002s	sys 0.016s
I20260709 13:19:06.757581 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.773046 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:06.776659 16736 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:19:06.793121 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 584) took 1119 ms. Trace:
I20260709 13:19:06.793720 17409 rpcz_store.cc:276] 0709 13:19:05.674108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.674279 (+   171us) service_pool.cc:224] Handling call
0709 13:19:05.677347 (+  3068us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.699496 (+ 22149us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:05.897073 (+197577us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.582012 (+684939us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.582228 (+   216us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.582781 (+   553us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.582951 (+   170us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:06.582997 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.583085 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.792824 (+209739us) raft_consensus.cc:1685] finished
0709 13:19:06.792852 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:06.793025 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21733,"prepare.queue_time_us":170296,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":349602,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":2205,"thread_start_us":452114,"threads_started":1,"wal-append.queue_time_us":744732}
I20260709 13:19:06.795557 20302 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:06.742302 19911 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:06.796672 20191 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:06.772342 20018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.002s	sys 0.016s
I20260709 13:19:06.825583 16740 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:19:06.643733 20204 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:19:06.685978 20299 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:06.692683 20225 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:06.687049 20240 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:06.771351 16589 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:06.826485 16738 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:06.737380 16708 leader_election.cc:403] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:06.851470 16745 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
W20260709 13:19:06.862147 17412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.022s	sys 0.038s
W20260709 13:19:06.964082 17412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.023s	sys 0.038s
W20260709 13:19:06.964191 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 227492us
I20260709 13:19:06.857383 20028 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:06.978237 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.966665 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:19:06.926527 16587 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.023s	sys 0.010s
W20260709 13:19:06.981927 16587 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.023s	sys 0.010s
W20260709 13:19:06.929904 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.016s	sys 0.023s
W20260709 13:19:06.933771 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 475527us
W20260709 13:19:06.983350 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.016s	sys 0.023s
W20260709 13:19:06.983647 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 776210us
I20260709 13:19:06.985528 16594 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:06.987396 17409 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 13:19:06.988399 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.988884 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:06.939066 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 495760us
W20260709 13:19:06.879908 20036 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.000s
I20260709 13:19:06.941573 16740 pending_rounds.cc:85] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 9
I20260709 13:19:06.990226 16740 pending_rounds.cc:107] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 10.10
W20260709 13:19:06.990443 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.990512 16740 raft_consensus.cc:2889] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:06.876987 20035 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:06.961752 20312 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.682s	user 0.000s	sys 0.013s
W20260709 13:19:06.968966 16636 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.045s	sys 0.049s
W20260709 13:19:07.019505 16636 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.045s	sys 0.050s
W20260709 13:19:07.019955 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 806113us
W20260709 13:19:06.974025 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:06.991545 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.026588 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:06.992856 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:06.993681 19577 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:07.027798 19577 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:19:06.925953 20038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.020s	sys 0.015s
W20260709 13:19:07.006852 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:06.957669 20125 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.030184 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:07.034713 17419 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:07.030964 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:07.037372 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:07.048116 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:06.977949 20211 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:06.991287 20304 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:06.977383 20028 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:06.991001 20209 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:06.977640 20080 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:06.994778 20017 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:07.061486 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.048116 19577 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:07.036464 20038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.020s	sys 0.015s
I20260709 13:19:07.034996 20312 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:07.088802 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:07.074283 20349 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:06.991429 20191 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:19:07.105453 20349 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.093144 16736 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:07.107858 20349 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:07.108714 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 2064) took 1048 ms. Trace:
I20260709 13:19:07.109162 16671 rpcz_store.cc:276] 0709 13:19:06.060587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.060728 (+   141us) service_pool.cc:224] Handling call
0709 13:19:06.061122 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.061399 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.061435 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.071656 (+ 10221us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.681634 (+609978us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:19:06.681698 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.681780 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.108472 (+426692us) raft_consensus.cc:1685] finished
0709 13:19:07.108495 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.108650 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":553117,"threads_started":1,"wal-append.queue_time_us":963236}
W20260709 13:19:07.111527 20338 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 13:19:07.111280 20028 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.112389 20028 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:07.097086 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:07.099730 20218 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:07.100073 20340 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:19:07.017619 19911 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:19:07.117506 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 329) took 1128 ms. Trace:
I20260709 13:19:07.101071 20204 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.074805 20140 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:07.053130 20118 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:06.978433 20299 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.116971 20191 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:07.092851 20304 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:19:07.014819 20036 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.117908 16590 rpcz_store.cc:276] 0709 13:19:05.989296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.011210 (+ 21914us) service_pool.cc:224] Handling call
0709 13:19:06.011518 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:06.048888 (+ 37370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.048932 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:06.049572 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 13:19:06.384127 (+334555us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:06.459779 (+ 75652us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.460280 (+   501us) log.cc:844] Serialized 14 byte log entry
0709 13:19:06.460495 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.460601 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:06.785112 (+324511us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4800002958. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:19:07.117294 (+332182us) raft_consensus.cc:1685] finished
0709 13:19:07.117313 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.117447 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19716,"prepare.queue_time_us":76936,"raft.queue_time_us":20139,"raft.run_cpu_time_us":11993,"raft.run_wall_time_us":323842,"spinlock_wait_cycles":118299264,"thread_start_us":334170,"threads_started":1,"wal-append.queue_time_us":798137}
I20260709 13:19:07.120995 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:07.092070 16738 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:07.123979 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.118355 20140 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.092747 20211 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:07.117843 20336 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:19:07.074350 20205 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:07.114555 20239 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:19:07.090191 20018 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:07.050401 20125 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.138113 20140 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:07.145893 20087 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.847s	user 0.011s	sys 0.001s
I20260709 13:19:07.146136 20125 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.138527 20240 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:07.148744 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.149318 17420 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:07.137542 20113 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:19:07.154908 17375 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.019s	sys 0.011s
W20260709 13:19:07.155377 17375 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.020s	sys 0.012s
I20260709 13:19:07.157646 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 693) took 1198 ms. Trace:
I20260709 13:19:07.158087 17412 rpcz_store.cc:276] 0709 13:19:05.958916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.959062 (+   146us) service_pool.cc:224] Handling call
0709 13:19:05.959372 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.968154 (+  8782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.968183 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.978986 (+ 10803us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.964489 (+985503us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:06.964528 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.964593 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.154559 (+189966us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b48002316d8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:19:07.157339 (+  2780us) raft_consensus.cc:1685] finished
0709 13:19:07.157359 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.157581 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8412,"raft.queue_time_us":8967,"raft.run_cpu_time_us":42083,"raft.run_wall_time_us":1175517,"spinlock_wait_cycles":40558976,"thread_start_us":913262,"threads_started":1,"wal-append.queue_time_us":1104930}
I20260709 13:19:07.146402 20087 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.129757 20099 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:19:07.165971 16635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.008s	sys 0.019s
W20260709 13:19:07.166386 16635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.008s	sys 0.019s
I20260709 13:19:07.147933 20125 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.118934 20225 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:07.168584 16710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.009s	sys 0.018s
W20260709 13:19:07.168931 16710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.009s	sys 0.018s
W20260709 13:19:07.169951 16665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.020s	sys 0.013s
W20260709 13:19:07.170377 16665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.020s	sys 0.013s
I20260709 13:19:07.131882 20204 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.173256 16588 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:07.176432 17409 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:07.183601 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 2061) took 1129 ms. Trace:
I20260709 13:19:07.184317 16668 rpcz_store.cc:276] 0709 13:19:06.054575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.054674 (+    99us) service_pool.cc:224] Handling call
0709 13:19:06.054926 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.055106 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:06.678749 (+623643us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.183260 (+504511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.183308 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:07.183353 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.183430 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.183546 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62655,"prepare.queue_time_us":71593,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":1051775,"threads_started":2}
W20260709 13:19:07.174947 20341 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 13:19:07.210160 16737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.011s	sys 0.028s
W20260709 13:19:07.216879 16741 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.010s	sys 0.020s
W20260709 13:19:07.226213 16737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.011s	sys 0.028s
W20260709 13:19:07.226423 16741 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.011s	sys 0.020s
I20260709 13:19:07.227798 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2660) took 1259 ms. Trace:
W20260709 13:19:07.240329 16670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.004s	sys 0.018s
W20260709 13:19:07.240815 16670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.005s	sys 0.018s
W20260709 13:19:07.243824 16739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.018s	sys 0.029s
W20260709 13:19:07.244161 16739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.018s	sys 0.029s
I20260709 13:19:07.244597 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:07.246181 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:07.249933 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:07.228988 16741 rpcz_store.cc:276] 0709 13:19:05.968869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.969057 (+   188us) service_pool.cc:224] Handling call
0709 13:19:06.015063 (+ 46006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.036192 (+ 21129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.036230 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.036786 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.407385 (+370599us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:06.598302 (+190917us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.227239 (+628937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.227461 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.227521 (+    60us) raft_consensus.cc:1685] finished
0709 13:19:07.227542 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.227725 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22905,"prepare.queue_time_us":207638,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":67,"raft.queue_time_us":64668,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":18106,"thread_start_us":996619,"threads_started":2,"wal-append.queue_time_us":876019}
I20260709 13:19:07.183508 20223 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:07.203331 19911 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.210037 20352 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:19:07.170604 20125 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.274735 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:07.270445 20343 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:19:07.281164 16743 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.027s
W20260709 13:19:07.276441 16664 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.015s	sys 0.012s
W20260709 13:19:07.281838 16664 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.015s	sys 0.012s
W20260709 13:19:07.281559 16743 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.000s	sys 0.027s
I20260709 13:19:07.243393 20038 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:07.248961 20191 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:07.250909 20294 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:07.271493 20347 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:07.290207 20217 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.022s	sys 0.027s
I20260709 13:19:07.282830 20223 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:07.304082 20217 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.023s	sys 0.027s
I20260709 13:19:07.289902 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 2053) took 1263 ms. Trace:
I20260709 13:19:07.276003 20017 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:19:07.306669 20348 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
W20260709 13:19:07.302920 20234 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.072s	user 0.007s	sys 0.006s
I20260709 13:19:07.339319 20234 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.340499 20223 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.341892 20234 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.316983 20087 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.317968 20211 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:07.319109 20346 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.180s	user 0.003s	sys 0.000s
I20260709 13:19:07.311669 20347 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.349830 20347 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.350215 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2657) took 1530 ms. Trace:
I20260709 13:19:07.309326 20036 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:07.327457 20204 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.329828 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 694) took 1370 ms. Trace:
I20260709 13:19:07.306993 20294 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.307636 20278 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:07.296023 20191 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.350646 16743 rpcz_store.cc:276] 0709 13:19:05.820304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.823114 (+  2810us) service_pool.cc:224] Handling call
0709 13:19:05.823418 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.022316 (+198898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.022348 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.066164 (+ 43816us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.628739 (+562575us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:07.289122 (+660383us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.290402 (+  1280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.318374 (+ 27972us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:07.335950 (+ 17576us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54003d3954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.336018 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.350002 (+ 13984us) raft_consensus.cc:1685] finished
0709 13:19:07.350021 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.350158 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":979,"prepare.queue_time_us":72816,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":5929,"raft.queue_time_us":43100,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":6047,"spinlock_wait_cycles":439571968,"thread_start_us":1065678,"threads_started":2,"wal-append.queue_time_us":945616}
W20260709 13:19:07.336846 20254 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.421s	user 0.013s	sys 0.005s
W20260709 13:19:07.337653 20354 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:19:07.283030 19911 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.368501 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2788) took 1171 ms. Trace:
I20260709 13:19:07.368904 19911 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.351212 16591 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:07.370186 20191 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:07.371243 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2793) took 1171 ms. Trace:
I20260709 13:19:07.371618 16591 rpcz_store.cc:276] 0709 13:19:06.199832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.302395 (+102563us) service_pool.cc:224] Handling call
0709 13:19:06.302742 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.370237 (+1067495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.371005 (+   768us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400320354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.371181 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2410757376}
I20260709 13:19:07.320158 20125 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.368927 16589 rpcz_store.cc:276] 0709 13:19:06.197508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.197626 (+   118us) service_pool.cc:224] Handling call
0709 13:19:06.197893 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.962770 (+764877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.368282 (+405512us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.368444 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317917440}
I20260709 13:19:07.352977 17410 rpcz_store.cc:276] 0709 13:19:05.959225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.959668 (+   443us) service_pool.cc:224] Handling call
0709 13:19:05.959957 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.961243 (+  1286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.961286 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.961719 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.102615 (+140896us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b18003120a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:19:06.685200 (+582585us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:06.685238 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.685306 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.074509 (+389203us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b18003120a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7557 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:07.329592 (+255083us) raft_consensus.cc:1685] finished
0709 13:19:07.329617 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.329767 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":929,"raft.queue_time_us":1014,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":28328,"spinlock_wait_cycles":24000512,"thread_start_us":582411,"threads_started":1,"wal-append.queue_time_us":955383}
I20260709 13:19:07.361016 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2791) took 1161 ms. Trace:
I20260709 13:19:07.363103 20093 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.363503 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 589) took 1156 ms. Trace:
I20260709 13:19:07.378221 17420 rpcz_store.cc:276] 0709 13:19:06.207041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.208085 (+  1044us) service_pool.cc:224] Handling call
0709 13:19:06.210782 (+  2697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.149906 (+939124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.363283 (+213377us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b54004e5554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.363445 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2157811968}
I20260709 13:19:07.356261 19758 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:07.368265 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:07.370939 16492 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 4 to 6, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 6 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:07.329952 20209 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:07.380053 20209 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:07.380947 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:07.376906 16596 rpcz_store.cc:276] 0709 13:19:06.199275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.206660 (+  7385us) service_pool.cc:224] Handling call
0709 13:19:06.207031 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.291117 (+ 84086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.291158 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:06.291228 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:06.291272 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.360776 (+1069504us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b54000d9dd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.360825 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.360961 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192821504}
I20260709 13:19:07.383736 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2794) took 1183 ms. Trace:
I20260709 13:19:07.384261 16588 rpcz_store.cc:276] 0709 13:19:06.200063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.369388 (+169325us) service_pool.cc:224] Handling call
0709 13:19:06.369791 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.174198 (+804407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.383421 (+209223us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54001faed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.383659 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847205248}
I20260709 13:19:07.405059 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:07.410923 16665 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:07.416359 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:07.417398 17410 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:07.406497 20211 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.423643 20211 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.426071 20211 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.427457 17411 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:07.428896 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:07.432957 16494 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 141) took 7500 ms. Trace:
I20260709 13:19:07.435191 16492 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 5, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 5 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:07.444114 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 451611us
I20260709 13:19:07.408550 20026 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:07.451515 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:07.452442 16593 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:07.451026 20026 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:07.457486 16492 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:07.461817 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.437541 16494 rpcz_store.cc:276] 0709 13:18:59.932784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:59.933160 (+   376us) service_pool.cc:224] Handling call
0709 13:19:00.379554 (+446394us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:19:07.432834 (+7053280us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:04.774973 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:04.775541 (+   568us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:04.775575 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:19:04.775593 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:04.776641 (+  1048us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:04.776698 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:04.776726 (+    28us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:19:04.777690 (+   964us) tablet.cc:701] Row locks acquired
0709 13:19:04.777706 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:19:04.777872 (+   166us) write_op.cc:270] Start()
0709 13:19:04.777982 (+   110us) write_op.cc:276] Timestamp: P: 1783603144777822 usec, L: 0
0709 13:19:04.778005 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:19:04.860059 (+ 82054us) log.cc:844] Serialized 4693 byte log entry
0709 13:19:05.247416 (+387357us) op_driver.cc:464] REPLICATION: finished
0709 13:19:05.469788 (+222372us) write_op.cc:301] APPLY: starting
0709 13:19:05.471907 (+  2119us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:05.494239 (+ 22332us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:05.504570 (+ 10331us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:05.532758 (+ 28188us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:05.533078 (+   320us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:19:05.533144 (+    66us) write_op.cc:312] APPLY: finished
0709 13:19:05.533480 (+   336us) log.cc:844] Serialized 83 byte log entry
0709 13:19:05.533645 (+   165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:05.533779 (+   134us) write_op.cc:454] Released schema lock
0709 13:19:05.533825 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:05.534267 (+   442us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:05.534297 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1951736,"reported_tablets":28,"spinlock_wait_cycles":1008790784,"child_traces":[["op",{"apply.queue_time_us":188401,"apply.run_cpu_time_us":4622,"apply.run_wall_time_us":76611,"num_ops":11,"prepare.queue_time_us":407555,"prepare.run_cpu_time_us":21067,"prepare.run_wall_time_us":160179,"raft.queue_time_us":179810,"raft.run_cpu_time_us":30652,"raft.run_wall_time_us":189203,"replication_time_us":436485,"thread_start_us":484611,"threads_started":4,"wal-append.queue_time_us":87545,"wal-append.run_cpu_time_us":52141,"wal-append.run_wall_time_us":1575937}]]}
I20260709 13:19:07.310642 16664 rpcz_store.cc:276] 0709 13:19:06.026786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.026999 (+   213us) service_pool.cc:224] Handling call
0709 13:19:06.027342 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.027551 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:06.089007 (+ 61456us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.289393 (+1200386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.289465 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:07.289531 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.289618 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.289829 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6559,"prepare.queue_time_us":98670,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":29002,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":14894,"thread_start_us":1193327,"threads_started":2,"wal-append.queue_time_us":946169}
I20260709 13:19:07.469048 16593 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:07.470832 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:07.471740 16593 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:07.399641 19758 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.443656 20299 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.412914 20254 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:19:07.478688 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.479328 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:07.481083 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 488) took 1019 ms. Trace:
I20260709 13:19:07.485896 16737 rpcz_store.cc:276] 0709 13:19:06.461873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.462525 (+   652us) service_pool.cc:224] Handling call
0709 13:19:06.462985 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:06.513314 (+ 50329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:06.513346 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:06.550569 (+ 37223us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.226719 (+676150us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:07.226759 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.226833 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.480804 (+253971us) raft_consensus.cc:1685] finished
0709 13:19:07.480832 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.481001 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65413,"raft.queue_time_us":87456,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":36178,"thread_start_us":559917,"threads_started":1,"wal-append.queue_time_us":838698}
W20260709 13:19:07.487371 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 450056us
I20260709 13:19:07.481931 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.477178 16664 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:07.410477 20093 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:07.504770 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:07.505379 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.474587 20125 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.498001 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.502324 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:07.513731 16492 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 3, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 3 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:07.503468 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.506130 16667 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:07.511643 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:07.481806 20126 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.479669 20067 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:07.493894 20087 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:07.454793 20026 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:07.491539 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 524889us
I20260709 13:19:07.581313 20026 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:07.581522 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:07.582728 20205 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:07.509006 20234 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:07.494316 20217 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:07.578682 20067 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.580155 20185 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:07.617636 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:07.583272 20205 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:07.576946 20339 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.616s	user 0.004s	sys 0.000s
I20260709 13:19:07.599761 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:07.670581 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53049.2us
I20260709 13:19:07.683550 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:07.685202 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85421us
I20260709 13:19:07.669466 20234 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.689126 20234 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:07.518893 20125 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.519258 20357 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.157s
I20260709 13:19:07.583915 20204 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:07.519025 20126 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.577950 20211 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:19:07.669785 20185 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:07.708431 16587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.038s	sys 0.043s
W20260709 13:19:07.692843 20140 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
W20260709 13:19:07.718268 16587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.038s	sys 0.043s
I20260709 13:19:07.719600 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:07.615911 20218 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 13:19:07.721369 20365 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:07.603700 20374 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.129s
I20260709 13:19:07.615738 20087 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.674762 20217 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.668514 20239 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:07.675814 20223 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:19:07.747550 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2659) took 1798 ms. Trace:
I20260709 13:19:07.721169 20140 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.777134 16742 rpcz_store.cc:276] 0709 13:19:05.949217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:05.954358 (+  5141us) service_pool.cc:224] Handling call
0709 13:19:05.954718 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:05.959001 (+  4283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:05.959047 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:05.959667 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:19:06.377806 (+418139us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:06.377867 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:06.377974 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:07.419761 (+1041787us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:07.727222 (+307461us) raft_consensus.cc:1685] finished
0709 13:19:07.727253 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.747444 (+ 20191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19319,"raft.queue_time_us":5745,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":24877,"spinlock_wait_cycles":24955648,"thread_start_us":417668,"threads_started":1,"wal-append.queue_time_us":914629}
W20260709 13:19:07.708504 20003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.051s	sys 0.067s
I20260709 13:19:07.790422 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2786) took 1598 ms. Trace:
I20260709 13:19:07.790983 16587 rpcz_store.cc:276] 0709 13:19:06.192006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.192193 (+   187us) service_pool.cc:224] Handling call
0709 13:19:06.192490 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.192680 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:06.982900 (+790220us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.789990 (+807090us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.790070 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:07.790130 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.790213 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:07.790353 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64962,"prepare.queue_time_us":105083,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"thread_start_us":1468293,"threads_started":2}
I20260709 13:19:07.750113 20184 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:07.748219 20284 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:19:07.797554 16635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.011s	sys 0.013s
W20260709 13:19:07.823506 16635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.012s	sys 0.013s
I20260709 13:19:07.648867 16492 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 5 to 2, term changed from 6 to 4, leader changed from <none> to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from VOTER to NON_VOTER. New cstate: current_term: 4 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:07.751772 20113 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:19:07.831833 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.031s	sys 0.014s
W20260709 13:19:07.832183 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.031s	sys 0.014s
W20260709 13:19:07.832778 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 558101us
I20260709 13:19:07.833715 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:07.710528 20204 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:07.797272 20003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.052s	sys 0.067s
W20260709 13:19:07.795971 20028 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.002s
I20260709 13:19:07.750258 20234 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:19:07.847168 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:07.848345 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:07.750545 19804 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.075s
W20260709 13:19:07.857642 16745 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.011s	sys 0.025s
I20260709 13:19:07.788925 20370 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:07.868297 16745 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.012s	sys 0.025s
W20260709 13:19:07.836277 20018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.016s	sys 0.032s
I20260709 13:19:07.827069 20347 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.874758 16662 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:07.827611 20284 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:07.830672 20184 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:07.781386 20365 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.839249 20299 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:07.769555 20205 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:07.937263 16492 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 4 to 6, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:07.905184 20020 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:19:07.851797 20376 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:07.835482 20233 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.010s	sys 0.025s
W20260709 13:19:07.940346 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 93169.7us
I20260709 13:19:07.883570 20112 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:07.881987 20225 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
W20260709 13:19:07.904206 20018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.016s	sys 0.033s
I20260709 13:19:07.857765 20191 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:07.907825 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 59599.2us
I20260709 13:19:07.977335 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:07.956847 20269 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:07.872839 20113 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:07.876530 20375 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:07.877107 19758 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:07.859014 20254 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:19:07.966614 20093 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.968255 16492 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 4, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 4 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:07.969913 16670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.019s	sys 0.029s
W20260709 13:19:07.986547 16670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.020s	sys 0.029s
I20260709 13:19:07.915947 20284 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:19:07.972599 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 286455us
I20260709 13:19:07.926658 20184 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:07.939859 20218 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:07.989194 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:07.965128 20233 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.010s	sys 0.026s
W20260709 13:19:07.980734 20209 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.000s
I20260709 13:19:07.911568 20294 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.829843 20185 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.941412 19911 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:07.923179 20367 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:19:07.882642 20372 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 13:19:07.981132 20269 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:07.984198 20018 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:07.940110 20299 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.928349 20347 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.995416 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:07.996105 16595 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.035s	sys 0.031s
I20260709 13:19:07.982826 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:08.025856 16595 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.035s	sys 0.032s
W20260709 13:19:08.026456 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43679.4us
I20260709 13:19:08.027927 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:08.029320 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:07.993953 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 243508us
I20260709 13:19:08.030974 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:07.981958 20067 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:07.994412 19577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.008s	sys 0.012s
I20260709 13:19:08.015553 20140 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:07.998931 20125 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.966974 20376 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:08.041976 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 427100us
I20260709 13:19:08.042378 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:08.032999 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.035120 19577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.009s	sys 0.012s
I20260709 13:19:08.037128 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.002952 20126 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:07.993799 20374 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:19:08.055960 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.002697 20218 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:07.983301 20375 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:08.014736 20269 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.040719 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.065305 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.058384 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.040612 20185 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:08.066591 16592 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:08.051998 19911 raft_consensus.cc:2764] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:19:08.051029 20087 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.003950 20209 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.054198 20217 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.067534 16492 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 6, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:08.093528 16492 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:08.058827 19758 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:08.056670 20125 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:08.101039 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.095489 16492 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 2 to 6, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:08.101743 16667 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:08.090376 20391 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:08.105048 20382 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:08.105381 20391 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:08.106019 20383 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:19:08.106364 20391 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.069784 20126 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:19:08.076349 20378 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:08.104278 20375 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.060585 20377 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:19:08.118453 20365 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.120613 20392 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.137373 20375 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:08.138826 20218 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.055943 20113 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.139626 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2802) took 1152 ms. Trace:
I20260709 13:19:08.140066 16594 rpcz_store.cc:276] 0709 13:19:06.987065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.987188 (+   123us) service_pool.cc:224] Handling call
0709 13:19:06.987431 (+   243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.437996 (+450565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.438028 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.443925 (+  5897us) log.cc:844] Serialized 26 byte log entry
0709 13:19:07.876511 (+432586us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:07.876583 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:07.876709 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.139395 (+262686us) raft_consensus.cc:1685] finished
0709 13:19:08.139416 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.139567 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6446,"raft.queue_time_us":47660,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":6389,"thread_start_us":792266,"threads_started":2,"wal-append.queue_time_us":599546}
I20260709 13:19:08.141799 20347 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.093924 20204 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.112726 20357 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.134778 20378 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:08.112273 20093 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:08.120746 20373 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:08.098145 19758 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:08.125036 20389 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:19:08.098701 20087 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:08.122720 20035 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:08.102991 20356 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.153120 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.148097 20378 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.158563 16671 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:08.149734 20369 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:19:08.147984 20093 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.162330 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:08.182022 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:08.207121 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 162892us
I20260709 13:19:08.209916 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.152832 19758 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:08.212118 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:08.221164 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.222743 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.225487 17410 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:08.228101 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46220.6us
W20260709 13:19:08.213958 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.214892 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 177828us
I20260709 13:19:08.215415 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.234601 16663 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:19:08.235571 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:08.219412 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.237532 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:08.157842 20035 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:08.233256 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.216538 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 2056) took 2186 ms. Trace:
I20260709 13:19:08.199580 20087 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:08.240309 16668 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:08.239653 20017 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:08.202064 20356 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:08.262652 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 197416us
I20260709 13:19:08.266031 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.269443 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.270162 16745 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:08.270466 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:08.270671 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.271375 16742 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.274816 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.276610 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.279858 16737 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:08.281623 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.284291 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.285038 16739 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:08.285794 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.286564 16737 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:08.289676 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.246315 16670 rpcz_store.cc:276] 0709 13:19:06.029832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.029949 (+   117us) service_pool.cc:224] Handling call
0709 13:19:06.030225 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:06.030418 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:06.661562 (+631144us) log.cc:844] Serialized 14 byte log entry
0709 13:19:07.688701 (+1027139us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b480021c258. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:19:08.215704 (+527003us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.216041 (+   337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:08.216122 (+    81us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:08.216190 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.216305 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.216458 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91859,"prepare.queue_time_us":316846,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":122444,"raft.run_cpu_time_us":2484,"raft.run_wall_time_us":344806,"spinlock_wait_cycles":19846016,"thread_start_us":1983264,"threads_started":4,"wal-append.queue_time_us":795552}
I20260709 13:19:08.291956 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.313751 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.255882 20299 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.318046 16742 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
W20260709 13:19:08.289767 20337 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 1.294s	user 0.000s	sys 0.007s
I20260709 13:19:08.260146 20374 consensus_queue.cc:1243] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:08.299530 16744 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:08.349080 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.347146 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.353363 16742 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:08.320115 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 336) took 1294 ms. Trace:
I20260709 13:19:08.297760 16737 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:08.333017 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.343781 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.355536 16738 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:08.288064 20391 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.261493 20017 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:08.357717 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:08.365839 16739 raft_consensus.cc:2376] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10. Re-sending same reply.
I20260709 13:19:08.364574 16634 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:08.320554 20113 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.291225 20394 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:08.299472 20395 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:08.333135 20148 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.304863 20217 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:19:08.364207 20372 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:08.374847 20017 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.403795 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:08.405653 20393 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:08.406101 20393 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.406459 20393 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:08.410635 20026 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.829s	user 0.012s	sys 0.001s
I20260709 13:19:08.399081 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.411607 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:08.412286 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:08.413022 16744 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:08.413791 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.388216 20391 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.415361 20240 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:08.415310 20391 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.416846 20240 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:08.417259 20240 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.417558 20391 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.418642 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:08.419531 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:08.419808 20392 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:08.420434 20392 consensus_queue.cc:1243] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:08.414731 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.378046 20376 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.378976 20394 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:08.399700 20384 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
I20260709 13:19:08.386608 20209 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.454849 20099 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:08.375914 20403 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 13:19:08.417601 20240 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.386085 20377 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:19:08.381012 20385 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:19:08.496294 20391 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.538483 20391 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.540663 20391 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.541832 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.497444 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.545251 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.546345 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.497735 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.547137 16588 raft_consensus.cc:2376] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7. Re-sending same reply.
I20260709 13:19:08.547976 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.548704 20299 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:08.548990 20269 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.549352 20299 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.550204 16744 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:08.498163 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.386907 20087 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.498286 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.555212 16591 raft_consensus.cc:2376] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7. Re-sending same reply.
I20260709 13:19:08.556072 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.556651 16591 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:08.394515 20148 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.499558 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.560755 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:08.504078 20405 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:19:08.504491 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:08.507848 16737 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:08.572060 19577 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.572971 19577 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:08.492782 20397 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:19:08.484160 20376 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.519134 20408 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.519229 20026 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.525913 20209 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.527040 20347 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:08.542703 20028 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:08.425120 20407 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:08.557526 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.498854 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.600903 16591 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:08.601661 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.601673 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.602325 16587 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.554643 16594 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:08.605240 16594 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:19:08.465837 20394 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.500269 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.563372 16589 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:08.485837 20099 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.576027 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:08.610517 17410 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:08.577044 17420 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:08.612347 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.613158 17419 raft_consensus.cc:2410] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:19:08.615463 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.616070 17419 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:19:08.573503 19577 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.515225 20003 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:08.581765 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.480726 20205 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:08.622048 16596 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:08.391593 20395 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.441468 20372 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:08.589388 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 85078.1us
I20260709 13:19:08.509312 20393 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.609624 16591 raft_consensus.cc:2410] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:19:08.587617 20379 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:08.620301 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 215071us
I20260709 13:19:08.631841 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:08.632511 17411 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:08.632630 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.513s (SENT) [suppressed 28 similar messages]
I20260709 13:19:08.596383 20374 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.594616 20113 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.599556 20028 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:08.637557 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.645190 17420 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:08.614369 20099 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.589246 20376 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:08.634446 20370 consensus_meta.cc:287] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.733s	user 0.005s	sys 0.010s
I20260709 13:19:08.621785 20003 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:08.646386 19577 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.671501 16587 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:08.595085 20026 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.673934 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.674651 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.595340 20209 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.592376 20408 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.633175 20356 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.683297 19577 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:08.684293 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.694168 16591 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:08.694809 19577 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.695363 16591 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:08.687158 20398 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.478s	user 0.004s	sys 0.000s
I20260709 13:19:08.691052 20026 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.609727 20347 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.678894 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.695149 17420 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:08.689316 16588 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.694237 20093 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.697150 20148 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.697695 19577 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.698170 16591 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.767436 19577 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:08.699427 20411 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 13:19:08.703248 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.703114 20028 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.704483 20376 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.707206 20299 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.717288 20209 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.672798 20370 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:08.673331 19758 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:08.695410 20184 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.707s	user 0.007s	sys 0.004s
I20260709 13:19:08.730130 20395 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.654493 20113 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.633414 20205 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.733160 20408 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.645541 20379 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.834673 20205 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.835034 20379 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.733966 20269 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:08.744658 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 324064us
I20260709 13:19:08.745719 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.753971 17411 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
W20260709 13:19:08.679666 20284 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.678s	user 0.016s	sys 0.000s
I20260709 13:19:08.836383 20393 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.836683 20284 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.836138 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:08.838857 20284 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.767549 16591 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.768040 19577 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.614919 20407 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.841962 19577 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:08.772083 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 139485us
I20260709 13:19:08.787022 20426 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:08.788547 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 114689us
I20260709 13:19:08.843842 20372 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.844528 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.790387 16588 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:08.772307 20377 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.813079 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.849943 16596 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:08.819144 16669 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:08.829265 20113 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.832588 19758 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.836135 16587 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:08.857545 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.819171 20028 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.839476 20284 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:08.850194 16708 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.589s (SENT)
I20260709 13:19:08.855381 19577 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.855880 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.861061 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:08.858317 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.861832 16587 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:08.738525 20017 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.859215 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:08.842432 20407 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:08.856043 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:08.862917 16708 leader_election.cc:336] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.610s (SENT)
I20260709 13:19:08.864429 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.864615 19577 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:08.865599 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:08.867738 16596 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:08.855494 20377 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.869593 20426 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:08.870446 20017 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.871052 16745 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:08.871697 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.872414 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:08.871454 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:08.873209 16664 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:08.874145 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.874888 16596 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:08.870590 20377 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.883692 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:08.825441 20184 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.884600 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:08.875612 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.906332 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.907423 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.907925 16596 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:08.908542 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:08.909010 16596 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.911980 16587 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:08.912762 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:08.913241 16587 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:08.809243 20003 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.870734 20225 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:08.917586 16591 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:19:08.918004 16591 pending_rounds.cc:85] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 7
I20260709 13:19:08.354775 16595 rpcz_store.cc:276] 0709 13:19:07.025810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.027626 (+  1816us) service_pool.cc:224] Handling call
0709 13:19:07.027972 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:07.360655 (+332683us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.360689 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:07.413041 (+ 52352us) log.cc:844] Serialized 26 byte log entry
0709 13:19:08.026239 (+613198us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:08.279220 (+252981us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.319718 (+ 40498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.319822 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:08.319873 (+    51us) raft_consensus.cc:1685] finished
0709 13:19:08.319891 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:08.320034 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28741,"raft.queue_time_us":166726,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":11193,"thread_start_us":1131725,"threads_started":3,"wal-append.queue_time_us":675442}
I20260709 13:19:08.918291 16591 pending_rounds.cc:107] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:19:08.865972 19758 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.887357 20378 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:08.891373 20017 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.887482 20426 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:08.922106 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.920956 20148 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:08.927150 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:08.927642 20435 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
I20260709 13:19:08.928036 17414 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:08.928996 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.929714 17416 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:08.932914 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.933705 17411 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:08.935211 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1442) took 1968 ms. Trace:
I20260709 13:19:08.935649 16662 rpcz_store.cc:276] 0709 13:19:06.966897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.967941 (+  1044us) service_pool.cc:224] Handling call
0709 13:19:06.968247 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.921761 (+953514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.934967 (+1013206us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b54003895d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:08.935148 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084256128}
I20260709 13:19:08.936404 20184 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.937546 20347 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.942687 20359 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:08.943081 20003 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:08.943555 20359 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.944613 20218 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:08.945158 20218 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:08.920974 20407 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:08.919704 20225 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:08.948727 20225 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:08.948117 20407 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.951771 20407 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.948040 20359 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:08.957981 20380 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.944s	user 0.002s	sys 0.004s
W20260709 13:19:08.959448 20436 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:19:08.964452 17419 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:08.963635 20035 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.966467 20035 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:08.946333 20218 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:08.969216 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.971709 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:08.972354 17412 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:08.972180 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.973922 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:08.923507 20099 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.975246 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:08.977589 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.923648 20378 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:08.979372 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:08.980152 17414 raft_consensus.cc:2410] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:19:08.917891 20240 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.873438 20423 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:08.879890 20284 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:08.987520 20099 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.880137 20209 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:08.986799 17412 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
W20260709 13:19:08.985702 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 61500.2us
I20260709 13:19:08.984937 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:08.989888 16738 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:09.003953 17410 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:08.980088 20378 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:09.011229 20432 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:19:09.013160 20394 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.990731 20209 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:08.984925 20126 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:19:09.015188 20209 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.015726 20099 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:09.016103 20099 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.016930 20099 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.991581 20423 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:08.991235 20018 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:09.018989 20099 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.019229 20018 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.020299 20423 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.021829 20018 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:08.991333 20429 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:09.022636 20429 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:09.023097 20429 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:09.023507 20431 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.243s	user 0.004s	sys 0.003s
I20260709 13:19:09.024449 20240 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:08.981689 20017 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.028525 17412 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:09.029544 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:09.026737 20017 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:08.987789 20284 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:09.034726 20425 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:19:09.039229 20209 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.042171 20209 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.046008 20289 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:09.046577 20289 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.022619 20225 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.047606 20289 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.048504 20218 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.048825 20289 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.049643 20225 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.050944 20394 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.065383 20429 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.072598 20429 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.102622 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:09.106195 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.107218 16587 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:08.918583 16591 raft_consensus.cc:2889] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.115607 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2813) took 1424 ms. Trace:
I20260709 13:19:09.116300 16593 rpcz_store.cc:276] 0709 13:19:07.690812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:07.691001 (+   189us) service_pool.cc:224] Handling call
0709 13:19:07.691400 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:07.818854 (+127454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:07.818904 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:07.818947 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:07.818977 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.114903 (+1295926us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400410d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:09.115020 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.115335 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268150656}
I20260709 13:19:09.117991 16591 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:19:09.120730 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:09.121184 16590 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:08.981060 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46341.6us
I20260709 13:19:09.123642 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:09.123967 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.125137 16595 raft_consensus.cc:2410] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:09.127302 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.127786 16591 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:09.124187 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.131263 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:09.131860 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:08.990340 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.157320 20218 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:09.155182 16593 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:09.158392 20028 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:09.160099 20225 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:09.163021 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:09.164575 16739 pending_rounds.cc:85] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:19:09.165526 17411 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:19:09.166556 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:09.166960 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.167551 20289 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.177845 16596 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:09.189136 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:09.199100 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.163698 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:09.178661 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:09.208101 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.165548 16736 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:09.196827 20028 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:09.210318 16591 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.210772 20028 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:09.212946 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 80303.2us
I20260709 13:19:09.217075 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.214498 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:09.219364 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 229555us
I20260709 13:19:09.220715 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.198823 16739 pending_rounds.cc:107] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:19:09.202996 20225 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:09.241986 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.248193 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.229805 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.226132 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.234318 16739 raft_consensus.cc:2889] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:09.233997 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.226300 20020 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:19:09.271754 20020 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:09.271865 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1444) took 1061 ms. Trace:
I20260709 13:19:09.272258 20020 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:09.272404 16669 rpcz_store.cc:276] 0709 13:19:08.210255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.210460 (+   205us) service_pool.cc:224] Handling call
0709 13:19:08.210814 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:08.851055 (+640241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.271225 (+420170us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400346554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:09.271599 (+   374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398359808}
I20260709 13:19:09.275506 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.274976 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.276494 16588 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.277787 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.279106 16592 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:09.280855 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.280973 20218 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.285519 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.287406 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.288945 16741 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:09.290323 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.282593 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.235379 20225 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.292218 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.231417 20099 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:09.293076 20099 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:09.293227 16736 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:09.293668 20099 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:09.294685 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.231063 20026 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:09.294936 16588 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:09.295646 20026 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.297049 16588 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:09.298097 20026 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.302292 16736 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:09.282126 20020 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.318966 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2694) took 1024 ms. Trace:
I20260709 13:19:09.319636 20020 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:09.319581 16740 rpcz_store.cc:276] 0709 13:19:08.292703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.292836 (+   133us) service_pool.cc:224] Handling call
0709 13:19:08.293314 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.293680 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.293706 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:08.537296 (+243590us) spinlock_profiling.cc:243] Waited 9.76 ms on lock 0x7b4c0002d640. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327deb0a20 00007f327dec1a29 00007f327de87644 00007f327deaef2d 00007f327deaeb93 00007f327fbdf18b 00007f327fbdfcb5 00007f327fbe1004 00007f327fbe0729 00007f327fbe5d96 00007f327fbdaadb
0709 13:19:08.538101 (+   805us) log.cc:844] Serialized 548 byte log entry
0709 13:19:08.800653 (+262552us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:08.800707 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:08.800802 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.226904 (+426102us) raft_consensus.cc:1685] finished
0709 13:19:09.226948 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.316676 (+ 89728us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22442752,"thread_start_us":262216,"threads_started":1,"wal-append.queue_time_us":366865}
W20260709 13:19:09.321709 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.298722 16588 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:19:09.307230 20289 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:09.309008 20419 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:09.227337 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.315351 17413 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:09.316072 20218 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.322408 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.314754 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:09.332244 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:09.343240 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.343771 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.343922 16596 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.344133 16736 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:09.344581 16741 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:09.351011 16742 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.010s
I20260709 13:19:09.351269 20419 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:09.351536 16742 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.004s	sys 0.010s
I20260709 13:19:09.351804 20419 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.353139 20419 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:09.355497 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 138633us
I20260709 13:19:09.355930 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:09.356165 20289 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:09.358006 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 166659us
I20260709 13:19:09.354604 20419 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.361446 20289 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:09.365546 20289 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.364964 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:09.363189 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.373262 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.372509 16737 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:09.361065 20218 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.377185 16591 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:09.377552 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.379882 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.380076 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.379136 20284 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:09.380760 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:09.380506 20028 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.381343 20284 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.383476 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.383963 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.384505 16667 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:09.385017 20284 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.387295 20026 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:09.391016 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 400666us
I20260709 13:19:09.392596 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.395159 20359 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.394289 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.376926 20218 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.401513 16743 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:09.401665 20118 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:09.402424 20118 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:09.403939 20118 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.406243 16737 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:09.407637 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.408640 16737 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:09.409963 20357 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.410149 16740 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 7: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:09.410511 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:09.410810 20357 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.412768 20223 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:09.414264 20357 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:09.415984 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60234.1us
I20260709 13:19:09.418076 20372 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:09.418789 20372 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.419204 19758 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:09.417418 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:09.418998 20407 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 2 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:09.420132 20372 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.422472 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.423202 16745 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:09.416066 20126 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:09.404984 20218 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:09.374773 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 60296.6us
I20260709 13:19:09.427786 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.430975 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.439659 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.446053 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.449360 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.450045 16743 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:09.452438 20225 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:09.452459 20445 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.169s	user 0.000s	sys 0.002s
W20260709 13:19:09.456537 20444 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:19:09.459427 20450 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.117s	user 0.000s	sys 0.002s
I20260709 13:19:09.461014 20223 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:09.461598 20223 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:09.426156 20126 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:09.468174 16745 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
W20260709 13:19:09.475878 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58627.3us
I20260709 13:19:09.478669 20359 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:09.481670 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:09.482969 20357 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:09.483825 20357 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.472141 20126 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.485505 20223 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.486347 20357 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:09.491191 20442 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.168s	user 0.004s	sys 0.000s
I20260709 13:19:09.496381 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.497192 16744 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:09.498353 20447 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:19:09.499311 16740 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:09.500517 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.501819 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:09.503228 20453 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.149s	user 0.000s	sys 0.002s
I20260709 13:19:09.472664 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1254) took 1174 ms. Trace:
I20260709 13:19:09.510638 16744 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
W20260709 13:19:09.511040 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86760.9us
I20260709 13:19:09.511226 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.515981 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:09.519785 16596 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:09.521133 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.467128 19758 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:09.529949 16741 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:09.450554 20099 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.536882 16744 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:09.491231 19911 raft_consensus.cc:2764] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:19:09.544075 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 72148.3us
I20260709 13:19:09.534613 20223 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.546411 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.547550 16741 raft_consensus.cc:2410] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:19:09.548892 16741 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:19:09.550985 16738 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.003s
I20260709 13:19:09.551427 20204 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:09.554128 20377 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:09.554324 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:09.555233 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:09.555794 17415 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:09.556667 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.556990 16670 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:09.557127 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 112361us
I20260709 13:19:09.557575 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.556923 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.528856 20218 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:09.563728 16664 raft_consensus.cc:2410] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:19:09.565454 16671 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:09.567801 16736 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:09.568885 20204 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.573079 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1449) took 1295 ms. Trace:
I20260709 13:19:09.573378 16662 raft_consensus.cc:2368] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:09.573997 20204 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.573599 16665 rpcz_store.cc:276] 0709 13:19:08.216626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.245539 (+ 28913us) service_pool.cc:224] Handling call
0709 13:19:08.245959 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:08.246409 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:08.304233 (+ 57824us) log.cc:844] Serialized 14 byte log entry
0709 13:19:08.980516 (+676283us) spinlock_profiling.cc:243] Waited 923 us on lock 0x7b180013c180. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f78423a 00007f327fbeea2a
0709 13:19:09.407389 (+426873us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:09.408359 (+   970us) log.cc:844] Serialized 52 byte log entry
0709 13:19:09.408581 (+   222us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:09.408617 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.408731 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.503287 (+ 94556us) raft_consensus.cc:1685] finished
0709 13:19:09.503402 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.511883 (+  8481us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15701,"prepare.queue_time_us":47354,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":17373,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":4593,"spinlock_wait_cycles":2123008,"thread_start_us":410801,"threads_started":1,"wal-append.queue_time_us":1166366}
I20260709 13:19:09.574823 17415 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:09.575428 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.581667 16666 raft_consensus.cc:2410] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:09.576038 16741 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:19:09.578945 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:09.582598 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.578267 16667 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:09.587496 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.545260 20028 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:09.588595 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 107096us
I20260709 13:19:09.509244 16742 rpcz_store.cc:276] 0709 13:19:08.298442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.505382 (+206940us) service_pool.cc:224] Handling call
0709 13:19:08.505788 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:08.506235 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:08.769501 (+263266us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.352188 (+582687us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:09.353113 (+   925us) log.cc:844] Serialized 52 byte log entry
0709 13:19:09.353268 (+   155us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:09.367930 (+ 14662us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.399076 (+ 31146us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.426474 (+ 27398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.432433 (+  5959us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.458606 (+ 26173us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:09.464288 (+  5682us) raft_consensus.cc:1685] finished
0709 13:19:09.464310 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.472549 (+  8239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34615,"prepare.queue_time_us":138892,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":133,"raft.queue_time_us":66723,"raft.run_cpu_time_us":531,"raft.run_wall_time_us":10841,"spinlock_wait_cycles":50120448,"thread_start_us":794138,"threads_started":2,"wal-append.queue_time_us":537146}
W20260709 13:19:09.595575 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.560258 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.551470 16738 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:09.529502 19758 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:09.605545 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.606791 16741 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:09.607517 20240 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:09.607856 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.629011 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:09.608295 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 2136) took 1107 ms. Trace:
I20260709 13:19:09.608637 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:09.609691 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.612746 16664 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:09.613781 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.579428 20396 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:19:09.620193 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:09.621953 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:09.633940 16737 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:09.644425 16666 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [5.4-8.6]
I20260709 13:19:09.644791 16664 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:09.647073 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:09.647756 16662 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:09.630035 19758 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:09.655505 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.657249 20240 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.657805 16667 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:09.658506 20240 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.660107 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.661257 20028 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:09.662210 16668 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:09.663154 20028 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:09.663756 20028 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:09.661656 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.665339 20028 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.666064 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:09.667979 16740 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:09.630467 16594 rpcz_store.cc:276] 0709 13:19:08.500882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.605920 (+105038us) service_pool.cc:224] Handling call
0709 13:19:08.606249 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.619571 (+ 13322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.619600 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:09.196483 (+576883us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b2c000014c8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:19:09.211439 (+ 14956us) log.cc:844] Serialized 26 byte log entry
0709 13:19:09.425028 (+213589us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:09.425080 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.441319 (+ 16239us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.607874 (+166555us) raft_consensus.cc:1685] finished
0709 13:19:09.607897 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.608114 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11031,"raft.queue_time_us":11789,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":20565,"spinlock_wait_cycles":25292672,"thread_start_us":213070,"threads_started":1,"wal-append.queue_time_us":394120}
W20260709 13:19:09.673549 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.677276 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:09.679817 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:09.682018 20020 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:09.682092 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.652051 20284 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:09.689177 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.689078 16741 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:09.621979 20126 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.670539 20225 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.691592 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:09.698204 16740 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:09.676081 20028 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:09.702404 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:09.741812 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.753675 16745 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:09.756138 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:09.770977 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 78515.7us
I20260709 13:19:09.671041 20289 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.781423 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.781845 20211 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:09.782351 20211 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.783169 20218 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:09.783672 20218 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:09.784096 20218 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.781514 20379 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:19:09.790093 20099 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:09.794538 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:09.795254 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:09.797508 20218 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:09.799675 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 112082us
I20260709 13:19:09.800310 16559 leader_election.cc:400] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:09.800755 16559 leader_election.cc:403] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:09.800949 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.802884 20184 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:09.782761 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.826282 16665 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:09.782750 20211 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:09.829106 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.833047 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.836674 20026 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:09.838057 20026 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:09.839399 20396 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:09.840322 20396 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.840623 16594 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:09.752681 20284 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.843441 20359 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:09.843855 20359 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:09.844898 20284 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:09.844839 20359 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:09.757999 20020 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.847913 20020 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:09.848227 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 2138) took 1346 ms. Trace:
I20260709 13:19:09.848986 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.855751 16670 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:09.858493 16742 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:09.861487 16592 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:09.826843 20126 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:09.841845 20396 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:09.838732 20026 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:09.848752 16589 rpcz_store.cc:276] 0709 13:19:08.501452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:08.611459 (+110007us) service_pool.cc:224] Handling call
0709 13:19:08.622370 (+ 10911us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:08.622786 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:08.622825 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:09.186232 (+563407us) spinlock_profiling.cc:243] Waited 9.15 ms on lock 0x7b2c000014c8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:19:09.209155 (+ 22923us) log.cc:844] Serialized 26 byte log entry
0709 13:19:09.585343 (+376188us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:09.595813 (+ 10470us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.602136 (+  6323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.656620 (+ 54484us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:09.836343 (+179723us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b1800358de0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7522 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:09.847817 (+ 11474us) raft_consensus.cc:1685] finished
0709 13:19:09.847846 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.848016 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3956,"prepare.queue_time_us":63680,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3238,"raft.queue_time_us":42075,"raft.run_cpu_time_us":548,"raft.run_wall_time_us":43686,"spinlock_wait_cycles":42074240,"thread_start_us":312611,"threads_started":1,"wal-append.queue_time_us":572693}
I20260709 13:19:09.872768 16741 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:09.878095 20018 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:09.879659 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:09.901391 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:09.887938 16587 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.011s	sys 0.026s
I20260709 13:19:09.903505 16745 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:19:09.903983 16587 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.012s	sys 0.026s
I20260709 13:19:09.902662 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:09.908089 20126 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:09.913806 16743 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:09.872155 20026 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:09.910445 20406 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:19:09.924620 16738 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [6.5-8.6]
W20260709 13:19:09.930836 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 114330us
I20260709 13:19:09.910805 20419 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:09.921535 20126 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:09.963912 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2698) took 1008 ms. Trace:
I20260709 13:19:09.968259 16744 rpcz_store.cc:276] 0709 13:19:08.955575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.543920 (+588345us) service_pool.cc:224] Handling call
0709 13:19:09.546815 (+  2895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.547201 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:09.547524 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 13:19:09.963395 (+415871us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:09.963474 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:09.963540 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:09.963638 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:09.963823 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3304,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1996,"thread_start_us":414586,"threads_started":1}
I20260709 13:19:09.946203 20419 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:09.920233 20377 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:19:09.984293 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:09.988343 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:09.962376 20233 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:09.989171 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:09.996428 16663 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.008s	sys 0.027s
W20260709 13:19:09.997139 16663 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.008s	sys 0.028s
I20260709 13:19:09.980113 20419 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.000560 20017 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:19:10.015722 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.029488 16590 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:10.013520 20118 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:10.018371 20289 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.022336 20357 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:10.053565 20003 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.053416 20394 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.031558 20218 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.066394 20372 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:10.036325 20017 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:10.074841 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 315367us
W20260709 13:19:10.073784 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 273780us
I20260709 13:19:10.054680 20357 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:10.086833 16594 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:10.094533 16588 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
W20260709 13:19:10.075352 20457 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:19:10.103282 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 514320us
I20260709 13:19:10.040561 20233 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:10.108073 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 144026us
I20260709 13:19:10.111827 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:10.054608 20026 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.128744 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2827) took 1354 ms. Trace:
I20260709 13:19:10.060675 20126 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.134907 20204 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:10.135840 20204 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:10.137199 20204 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:10.139338 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 434663us
I20260709 13:19:10.129554 16595 rpcz_store.cc:276] 0709 13:19:08.773968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.126168 (+352200us) service_pool.cc:224] Handling call
0709 13:19:09.126793 (+   625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:09.288339 (+161546us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:19:09.751491 (+463152us) log.cc:844] Serialized 14 byte log entry
0709 13:19:10.128264 (+376773us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:10.128344 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:10.128407 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.128495 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.128656 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244414,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":53,"thread_start_us":741047,"threads_started":2}
I20260709 13:19:10.145814 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:10.096072 20118 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.152782 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.111112 20020 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:19:10.156168 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 485630us
I20260709 13:19:10.163195 20357 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:10.170241 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.177480 20456 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:19:10.178344 20476 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:10.179373 20476 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:10.181352 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 98362.5us
W20260709 13:19:10.183064 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.167383 20018 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:10.191812 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.189146 20018 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:10.193593 20018 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.196758 20184 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:10.174031 20017 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.199342 20118 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:10.206811 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:10.203047 20099 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:10.223740 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:10.216871 20284 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:10.239193 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86504us
W20260709 13:19:10.241843 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 74339.2us
I20260709 13:19:10.234767 20284 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.250562 20469 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.251432 20469 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.251891 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:10.252674 16670 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.253774 20469 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.254398 20469 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.277169 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:10.266624 20377 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.278446 20377 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.279035 20470 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:10.281229 20377 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:10.281708 19758 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.591s	user 0.007s	sys 0.003s
I20260709 13:19:10.248268 20284 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.282814 16667 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:10.279126 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:10.283599 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:10.284054 16666 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.284354 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:10.285197 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:10.282346 19758 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.286741 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:10.287375 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1261) took 1328 ms. Trace:
I20260709 13:19:10.287868 16665 raft_consensus.cc:2410] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:19:10.287833 16736 rpcz_store.cc:276] 0709 13:19:08.959272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.601961 (+642689us) service_pool.cc:224] Handling call
0709 13:19:09.651211 (+ 49250us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:09.723674 (+ 72463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:09.723846 (+   172us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:09.772134 (+ 48288us) log.cc:844] Serialized 52 byte log entry
0709 13:19:09.822139 (+ 50005us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b180030e320. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:19:10.175580 (+353441us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:10.175624 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.175723 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:10.286971 (+111248us) raft_consensus.cc:1685] finished
0709 13:19:10.286997 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.287299 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1099,"raft.queue_time_us":12017,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":49018,"spinlock_wait_cycles":14581120,"thread_start_us":345877,"threads_started":1,"wal-append.queue_time_us":442150}
I20260709 13:19:10.289431 19758 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:10.290329 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.277525 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:10.294214 20469 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:10.297768 20469 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:10.298321 20466 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:19:10.298234 20469 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:10.299944 20469 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.301424 20469 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:10.291409 20459 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.117s	user 0.001s	sys 0.001s
W20260709 13:19:10.291342 20465 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:19:10.308585 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:10.310238 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52741.5us
I20260709 13:19:10.316253 16666 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:10.318441 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:10.321923 20211 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.323396 16589 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 13:19:10.324661 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49248.2us
I20260709 13:19:10.325519 16670 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
W20260709 13:19:10.330488 20462 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.166s	user 0.001s	sys 0.001s
I20260709 13:19:10.336838 20018 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.347132 20018 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:10.355422 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 166305us
I20260709 13:19:10.356429 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:10.346735 20028 consensus_meta.cc:287] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.001s
I20260709 13:19:10.364125 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.366498 20479 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:10.368964 20372 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:10.369185 20479 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:10.369451 20372 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.369886 20372 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:10.370268 20479 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.372521 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:10.375067 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 56717.1us
I20260709 13:19:10.375962 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.377244 20211 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.377923 20211 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:10.379137 20394 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:10.380169 20394 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:10.380538 20394 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:10.381352 20461 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.311s	user 0.000s	sys 0.002s
I20260709 13:19:10.381664 20394 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.382670 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2822) took 1712 ms. Trace:
I20260709 13:19:10.383083 16587 rpcz_store.cc:276] 0709 13:19:08.670015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.109131 (+439116us) service_pool.cc:224] Handling call
0709 13:19:09.110128 (+   997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:09.134031 (+ 23903us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:09.134065 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:09.136490 (+  2425us) log.cc:844] Serialized 548 byte log entry
0709 13:19:10.051860 (+915370us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:10.051931 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:10.052112 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:10.380516 (+328404us) spinlock_profiling.cc:243] Waited 6.09 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:10.382349 (+  1833us) raft_consensus.cc:1685] finished
0709 13:19:10.382368 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:10.382597 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14002944,"thread_start_us":767505,"threads_started":1,"wal-append.queue_time_us":933152}
I20260709 13:19:10.384999 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:10.386559 20379 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:10.388038 20284 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.388254 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:10.388309 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:10.390220 20394 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:10.394419 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.356981 20419 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.395998 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.394021 20020 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:10.405596 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44195.7us
I20260709 13:19:10.407138 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.407904 20483 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:10.408406 20483 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.408788 20483 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.409068 20471 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:10.409510 20471 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:10.410645 20284 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:10.411067 20464 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
W20260709 13:19:10.413813 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.404508 20379 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.009s
W20260709 13:19:10.416496 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50625.2us
I20260709 13:19:10.410436 20471 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.417920 17374 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.418329 20233 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:10.418737 20471 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.418906 20473 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:10.413415 20476 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.423498 20476 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.423967 20003 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.424156 20419 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.404698 20211 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:19:10.424625 20003 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.425786 20476 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.426169 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:10.426343 20476 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:10.427331 20379 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:10.427520 20469 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:10.434720 20357 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.439920 20493 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:10.442852 20479 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:10.441764 20211 consensus_queue.cc:1243] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:10.426751 20003 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.443396 20480 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:19:10.448652 20482 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
W20260709 13:19:10.449522 20467 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.305s	user 0.003s	sys 0.000s
I20260709 13:19:10.437389 20473 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:19:10.464600 17413 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:10.465011 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:10.472498 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69031.9us
I20260709 13:19:10.477442 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.479799 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:10.444332 20493 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.486878 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:10.444988 20479 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:19:10.497380 20486 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:19:10.517508 16594 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:10.515852 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:10.520051 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:10.523168 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.494807 20482 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:10.526330 20482 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.519510 20284 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:10.494494 20396 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:10.527254 20396 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.527336 20284 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:10.527710 20396 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.528350 20284 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.532069 20484 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:10.532750 20484 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:10.533846 20484 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.536479 16737 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:19:10.537546 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.538096 20225 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:10.538753 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.539420 16587 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:10.526742 20482 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:10.539871 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.485808 20372 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.540992 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:10.541378 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.541906 16737 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:10.542172 20372 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.542495 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:10.542188 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.540288 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:10.539242 20377 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:10.545097 20377 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:10.545994 20377 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.547209 20377 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:10.553870 20483 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.495347 20493 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:10.557374 20233 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:19:10.560873 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.572815 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.573554 16745 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:10.499847 20480 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.575392 20480 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.578197 20480 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:10.578992 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52288.1us
I20260709 13:19:10.582715 20218 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.583670 16559 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.583530 20218 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.584908 20211 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.585706 20218 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.499332 20289 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:10.586262 20289 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.586640 20289 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.586738 20394 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:10.587010 20469 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.587678 20469 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.587978 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.588841 16595 raft_consensus.cc:2368] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:10.589879 20469 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.581123 20479 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.591095 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.591601 20479 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:10.592278 16595 raft_consensus.cc:2410] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:19:10.585469 20211 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.582688 16736 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:10.592870 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.602840 16590 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:10.603241 16745 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:10.602847 16589 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:10.579597 20473 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:19:10.595396 16737 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:10.624594 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.625831 16743 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:10.627103 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:10.632665 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.636610 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.638638 16741 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:10.640576 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.643345 16740 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:10.643924 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.644680 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.645980 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.646463 16745 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:10.646548 16744 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:10.648232 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 217139us
I20260709 13:19:10.666908 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.667773 16740 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.671198 16742 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:10.675514 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:10.681838 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.681998 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:10.683069 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:10.629910 20480 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:10.682700 16743 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:10.679905 16594 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 2 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:10.694839 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 254814us
I20260709 13:19:10.705870 16741 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:19:10.648873 20357 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.708691 16744 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:10.723261 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:10.673405 20469 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.696210 20218 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.739066 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:10.702351 20485 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:10.748780 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.751394 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59235.4us
I20260709 13:19:10.752489 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.747537 16588 pending_rounds.cc:85] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 5
I20260709 13:19:10.754187 16588 pending_rounds.cc:107] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:19:10.754659 16588 raft_consensus.cc:2889] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:10.754909 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49452.7us
I20260709 13:19:10.727259 20484 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:19:10.778784 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:10.749650 20218 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.785416 20218 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.788334 16587 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:10.793192 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.793216 16593 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:19:10.794329 16593 pending_rounds.cc:85] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 5
I20260709 13:19:10.794656 16593 pending_rounds.cc:107] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:19:10.735984 20357 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.795401 16593 raft_consensus.cc:2889] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:10.810328 16738 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:10.736281 20480 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.765187 20485 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:10.742202 20469 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.753002 20471 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:10.826035 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.838419 20284 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.838548 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:10.727357 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:10.843653 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:10.766646 20500 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:10.849824 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 116267us
I20260709 13:19:10.793529 20359 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.857312 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:10.856513 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.856320 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:10.827692 20406 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:10.788326 20223 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.858431 16590 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:10.859668 16738 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:19:10.867183 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:10.868968 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.869788 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.857986 17412 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:10.864030 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:10.850664 20483 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.850147 20394 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:10.862058 20406 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.846218 20289 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.870565 20377 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:10.842991 20396 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.898226 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.885975 20003 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:10.895027 20211 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.905184 20240 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:10.897856 20406 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:10.909425 20396 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.909852 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:10.911031 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:10.913337 20501 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:19:10.890949 20502 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:19:10.889784 20204 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:10.861914 20284 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.908042 20394 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.921227 16596 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:19:10.921705 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.921790 20492 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:10.880832 20223 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.925282 16740 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:10.926530 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.928347 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:10.945292 17415 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:10.945170 20492 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.946132 20284 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.946761 20028 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:10.947227 20028 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:10.947590 20492 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:10.948134 20028 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.948770 20480 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.949478 20028 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:10.929065 16671 raft_consensus.cc:606] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:10.951495 16671 raft_consensus.cc:614] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:10.959290 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:10.960117 17416 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:10.901980 20377 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:10.961148 20377 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:10.961282 16595 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:10.954349 20204 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.963909 20204 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:10.967049 20482 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.967118 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:10.968943 16588 raft_consensus.cc:2376] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10. Re-sending same reply.
W20260709 13:19:10.969152 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:10.970001 20482 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:10.934571 20240 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:10.934988 20485 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:10.972493 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:10.978107 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:10.978102 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:11.012089 20492 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:11.013042 20492 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.013640 20492 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:11.016899 20223 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:11.020157 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:11.022652 20357 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.023244 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.023551 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.024121 16739 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:11.025787 20240 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:11.026129 20469 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:11.027256 20026 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:11.030612 17410 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:11.032581 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.034483 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.040787 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:11.042016 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68138.4us
I20260709 13:19:11.042243 16594 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.050670 16742 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:19:11.052496 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.053525 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.065063 16739 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:11.071961 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.073226 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.074314 16595 raft_consensus.cc:2410] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:19:11.073568 20223 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.083609 20223 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.084331 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.085481 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.086141 16593 raft_consensus.cc:2368] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9.
I20260709 13:19:11.072829 20501 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:19:11.088567 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.090461 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:11.055984 20493 consensus_meta.cc:287] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.000s
I20260709 13:19:11.055675 20017 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:11.095011 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48239.8us
I20260709 13:19:11.104342 16596 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:11.097664 20184 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:11.052095 17416 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 2 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:11.099815 20484 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:11.100632 20204 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:19:11.102137 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 61436.4us
I20260709 13:19:11.113225 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.121817 16596 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:11.123251 16591 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:11.085719 20223 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:11.105902 20479 consensus_meta.cc:287] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 13:19:11.125420 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:11.139007 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:11.139566 16595 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:11.131711 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:11.115272 20419 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.002s	sys 0.026s
I20260709 13:19:11.141332 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.144080 17413 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:11.146327 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.075227 20377 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:11.141821 20419 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.003s	sys 0.026s
I20260709 13:19:11.152350 20512 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.138958 20469 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:11.174335 20469 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.112061 20017 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.184763 20017 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:11.187026 20377 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.187333 20495 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:11.187873 20495 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:11.188270 20495 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:11.189354 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57816.9us
I20260709 13:19:11.190279 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:11.191716 20485 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:11.191711 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:11.192950 20470 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:11.142978 20289 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.193686 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:11.193861 20470 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:11.194311 20470 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:11.194993 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.195700 20028 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:11.195258 20470 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.196187 20028 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:11.196635 20028 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:11.196666 20470 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.197084 20377 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:11.197471 20377 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:11.111864 20493 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.198423 17411 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:11.198319 20377 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.199357 20482 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:11.153218 20406 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.199993 20482 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:11.200556 20482 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:11.169145 16592 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:19:11.201761 20406 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:11.192057 20473 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:11.195833 20480 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:11.160224 20479 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:11.202548 20480 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:11.203034 20480 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:11.203105 19911 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:11.203711 19911 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.204283 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.207039 16737 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:11.207973 16740 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:19:11.199522 20493 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.209551 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.206065 19911 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:11.210335 20492 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.211278 20479 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:11.211735 20479 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:11.212425 20492 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.215960 16708 leader_election.cc:400] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:11.216277 16708 leader_election.cc:403] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:19:11.217957 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:11.218976 20379 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:11.219722 20379 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.224524 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:11.225085 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 50673.5us
I20260709 13:19:11.232574 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.233350 20480 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.233919 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.234144 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:11.251194 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.234486 16595 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:11.235230 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.233027 20495 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.247067 20469 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.166772 16594 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:11.251384 16741 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.252377 16743 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
W20260709 13:19:11.235982 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48955.1us
I20260709 13:19:11.253166 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.268606 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:11.271554 16740 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:11.272045 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.274652 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:11.251212 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 154171us
I20260709 13:19:11.279783 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:11.281466 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57025.4us
I20260709 13:19:11.282266 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:11.292886 16666 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:11.293119 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.281068 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.295749 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:11.298355 16671 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:11.299046 17415 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.299698 16666 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:11.272233 20026 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:19:11.276081 19911 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:19:11.300621 20471 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:11.300859 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.300999 20471 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:11.301201 20407 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:11.287660 20469 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.302038 20471 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.286026 20359 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:11.303323 20471 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:11.304441 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.304749 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.296288 20512 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:11.309475 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.311595 20510 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.303093 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:11.315801 16594 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:11.317516 16662 raft_consensus.cc:2410] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:19:11.318120 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:11.318722 16591 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:11.320696 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:11.322602 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.322808 16741 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:11.323637 16670 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
W20260709 13:19:11.332851 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 226150us
I20260709 13:19:11.334443 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.337256 16595 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:11.334969 16588 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: [10.9-11.10]
I20260709 13:19:11.277016 20473 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:11.336141 16594 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:11.339205 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:11.340611 16593 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:11.285933 20233 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:11.315536 20211 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:11.342450 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:11.347524 16588 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:11.299798 20495 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:11.328348 20396 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:11.330763 20500 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:11.350437 16587 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:11.361413 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.353652 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:11.362038 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 169511us
I20260709 13:19:11.330718 20476 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:11.360716 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.362098 16670 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:11.364418 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:11.365381 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 81623us
I20260709 13:19:11.366408 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:11.348106 20020 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:11.324090 20511 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:19:11.362991 16662 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:11.382239 16739 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:19:11.321681 20479 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.384017 16589 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:11.385272 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:19:11.387553 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48177.1us
I20260709 13:19:11.394542 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:11.399451 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55890.2us
I20260709 13:19:11.387688 16591 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
W20260709 13:19:11.402161 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61674.2us
I20260709 13:19:11.359532 20211 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:11.410028 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.406282 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:11.411728 16745 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:11.411082 16662 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 5.
I20260709 13:19:11.413234 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:11.413535 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:11.379945 20233 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.417110 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.417757 16670 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:11.387017 20500 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:11.435724 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.410498 20476 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:11.410347 20396 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:11.415989 20379 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.428452 20407 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:19:11.465260 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62198.4us
I20260709 13:19:11.431106 20512 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.475029 16669 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:11.412599 20484 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:19:11.430915 20026 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:11.472560 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:11.451244 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:11.416124 20020 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:11.484532 20020 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.488778 16670 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:11.490774 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.491151 20500 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.491542 20511 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:11.492884 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.493350 20511 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:11.494232 20511 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:11.494586 20511 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:11.495050 20483 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:11.495608 20483 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:11.496470 20483 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.497682 20483 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:11.499750 20511 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.501325 20511 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:11.502101 20396 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.503150 20211 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.504431 16664 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:11.493568 16665 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
W20260709 13:19:11.500406 20517 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:19:11.490195 16670 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:11.477483 20522 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:19:11.481349 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 86884.6us
I20260709 13:19:11.519299 16593 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:11.520140 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:11.521169 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.491206 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.509760 16588 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:11.510568 16745 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:11.512333 20476 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.514061 20482 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.523854 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.444495 20028 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.471485 20407 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.529047 20407 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:11.521946 16668 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:11.516238 20480 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:11.527769 20469 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:11.483100 20026 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:11.478508 20419 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:19:11.530283 16595 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
W20260709 13:19:11.530892 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58370.2us
I20260709 13:19:11.565075 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:11.531553 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 86499.4us
I20260709 13:19:11.526599 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:11.541949 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:11.528262 20473 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:11.554391 16741 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:11.530591 20512 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.567191 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:11.573627 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 161204us
I20260709 13:19:11.576644 16665 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:11.579538 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:11.541846 20513 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:11.542399 20289 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:11.582543 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:11.581039 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45691.1us
I20260709 13:19:11.578534 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:11.582262 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.582662 20379 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.604475 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:11.594205 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:11.610704 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:11.601954 16592 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:11.606658 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:11.574288 20233 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:11.577870 20495 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:11.577936 20394 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:11.620415 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.634088 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:11.634338 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.582345 20480 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:11.604993 20003 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:11.643821 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.601136 20513 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:11.600428 20423 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:11.655591 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49018.7us
I20260709 13:19:11.648276 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.656458 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:11.655059 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:11.613634 20512 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:11.618342 20522 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:11.603036 20289 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:11.621063 20473 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:11.660732 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:11.662410 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 348177us
I20260709 13:19:11.576783 20524 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:19:11.643759 20480 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.636874 20394 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:11.586318 20519 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:19:11.658633 20522 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:11.739993 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.728292 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:11.698541 20003 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:11.702376 16588 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:11.617985 20379 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:11.704613 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:11.750789 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:11.662431 20531 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:11.807629 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 79463us
I20260709 13:19:11.651221 20020 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:19:11.632818 20233 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.700320 20028 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:11.659314 20512 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:11.731721 20423 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:11.732856 20184 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.815964 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:11.737453 20480 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:11.607156 20026 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.738075 20394 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:11.661846 20495 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:11.829242 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:11.754328 20522 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:11.809124 20471 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:11.835843 16663 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:11.815845 20500 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.809870 20479 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:11.864687 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:11.866079 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 209716us
I20260709 13:19:11.821010 20020 consensus_queue.cc:1243] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:11.821058 20211 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.815506 20531 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:11.831475 20512 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:11.908277 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:19:11.953483 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 45302.7us
I20260709 13:19:11.872414 20513 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.869028 20493 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:11.889328 20028 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:11.882961 20520 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:19:11.939257 20535 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:19:11.934973 20357 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:11.914338 20492 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:11.909888 20471 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:11.938284 20377 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:12.013856 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 184704us
I20260709 13:19:11.914036 20525 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:11.972059 20547 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:12.024863 20547 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:12.025902 20547 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.926760 20003 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.968626 20531 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.020464 20548 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:11.952416 20473 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:12.031728 20473 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.886750 20532 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.408s
I20260709 13:19:11.939733 20423 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.015491 20357 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:12.050377 20538 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:19:12.050509 20471 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:12.054565 20534 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
W20260709 13:19:12.056135 20540 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:19:12.054339 20357 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:11.968365 20211 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.057850 20357 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.059273 20233 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.059697 20211 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:12.060407 20544 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
W20260709 13:19:12.062731 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.067493 20558 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.068136 20558 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.005815 20493 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:11.988621 20483 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:19:11.893646 20289 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.045079 20492 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:11.994902 20028 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.045464 20482 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.070726 20558 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:12.111220 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 295365us
I20260709 13:19:12.119942 17374 leader_election.cc:400] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9.
W20260709 13:19:12.112582 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 284066us
W20260709 13:19:12.110448 20511 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
W20260709 13:19:12.123070 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 168282us
I20260709 13:19:12.015144 20495 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.139865 16667 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:12.141485 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.150085 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.153631 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.154333 17416 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:12.160001 20539 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:19:12.163796 20407 consensus_meta.cc:287] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.591s	user 0.008s	sys 0.003s
I20260709 13:19:12.164317 20407 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.166530 20407 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.139904 20233 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:19:12.168560 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.185500 16591 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:12.185333 20492 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.187222 20511 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.168699 20550 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.173062 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.189507 17411 raft_consensus.cc:2410] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:19:12.173727 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.174244 20476 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:19:12.174583 20530 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.437s	user 0.003s	sys 0.000s
I20260709 13:19:12.175467 16737 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:12.176016 20493 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.178321 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.183220 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.105702 20531 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.187909 20511 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.205890 20554 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:19:12.206434 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.209270 17410 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:12.194725 17416 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:12.216919 20559 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.220854 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.221369 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.221930 17411 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:12.178663 20480 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:12.226780 20556 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:19:12.228097 20233 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:12.228511 20233 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:12.229360 20233 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.219502 20017 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.236030 16662 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:12.226534 20555 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.231312 20480 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.238262 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.238991 17416 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:12.239439 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.208156 20525 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:12.241278 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.243294 20557 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.230696 20233 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.251494 17410 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:12.229982 20359 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.231505 20511 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.253382 20020 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.256263 20020 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.271288 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:12.222121 20559 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.251550 20017 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.277070 20017 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.281847 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.282509 17415 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:12.280159 20394 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.283435 20531 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.283921 19758 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:12.285300 20536 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.396s	user 0.002s	sys 0.000s
I20260709 13:19:12.208418 20550 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.287955 20522 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.257973 20557 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.269205 20359 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.295658 17411 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 6.
I20260709 13:19:12.309953 16588 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:19:12.310709 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.229403 20512 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.314110 17410 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:12.318476 20379 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.252425 20555 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:12.328729 20568 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:12.331936 20512 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.328430 19911 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:12.337567 20493 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:12.338003 20493 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:12.338306 19911 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:12.338709 19911 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:12.277070 20559 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:12.335922 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.336882 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:12.337039 20522 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.334287 20359 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.334682 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:12.305819 20017 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.365443 16594 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.365483 16596 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:12.349066 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.367100 16587 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:12.374344 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.376953 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.347219 20471 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.386937 20471 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.389423 20471 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.392764 17419 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:12.394949 17420 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:12.396555 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:12.397090 20550 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.397313 17414 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.399176 20557 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.402129 20394 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.359525 20548 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:12.345206 20028 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.412338 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:12.340126 20020 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.418483 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.419452 17411 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:19:12.373375 20359 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.424739 20359 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.413177 20394 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:12.438266 20567 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:19:12.441828 20548 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.445676 20555 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:12.450604 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.338807 20493 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.452152 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.454035 16594 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:12.455348 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:12.489606 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 218391us
I20260709 13:19:12.423125 20495 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:12.491873 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.493824 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:12.494930 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:12.496678 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.497064 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.497713 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.498184 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:12.495091 16663 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:19:12.499001 16663 pending_rounds.cc:85] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 3
I20260709 13:19:12.339819 19911 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.502400 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:12.499246 16663 pending_rounds.cc:107] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:19:12.502990 16663 raft_consensus.cc:2889] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:12.499905 16668 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:12.500232 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:12.500460 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.502051 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.451532 20218 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:12.380018 20017 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.502044 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.509238 17420 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:12.505916 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.509789 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.491586 20525 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.515350 16740 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:12.513713 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:12.493873 20211 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:12.516763 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:12.519290 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:12.521083 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.503264 20555 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:12.507107 20028 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:12.504786 20531 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:12.489413 20394 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.526616 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.506666 20546 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.506376 19758 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:19:12.508886 20493 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:12.518769 20557 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:12.531126 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.545982 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.531729 16591 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:12.524119 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:12.532254 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:12.508402 20204 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.510126 20407 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.504413 20532 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
W20260709 13:19:12.541242 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:12.525358 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.536026 20550 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:19:12.550427 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.558598 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.559913 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.564659 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.566898 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:12.543159 20546 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.515363 20218 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.551324 20289 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:12.551141 20204 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.551992 20407 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.531060 20512 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:12.546712 16663 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:12.536095 20531 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.593524 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:12.599242 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 67053.7us
I20260709 13:19:12.600358 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:12.610678 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.592837 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:12.611404 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:12.615334 16663 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:12.567837 20483 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.617703 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.537133 20492 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:19:12.523303 20525 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.621493 16741 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:19:12.621622 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:12.622704 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.639730 16744 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 8: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:12.621538 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:12.630456 17414 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:12.631974 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:12.641451 16742 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:12.624950 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.635866 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.621845 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.642930 16743 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:12.592715 20531 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:12.630576 20479 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:12.645144 20359 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:12.648296 20514 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.300s	user 0.011s	sys 0.000s
I20260709 13:19:12.591578 20289 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:12.652009 20017 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.651741 20289 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.653801 20218 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.655073 20543 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:12.655689 20543 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:12.656629 20543 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.659323 20571 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:12.660879 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.661712 16591 raft_consensus.cc:2410] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:12.662492 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:12.665813 20575 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:19:12.667095 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.667440 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.667881 20471 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:12.668300 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:12.668502 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.669374 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.635053 20483 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.672048 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.672400 20545 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.673005 20545 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.673874 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:12.675100 20551 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:12.621196 20579 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.676067 20579 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.641739 20559 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:12.679812 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:12.604269 20512 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:12.680385 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46338.4us
I20260709 13:19:12.681229 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.688983 16591 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:12.691469 16591 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:12.617183 20211 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.692270 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:12.695300 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:12.693073 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:12.678831 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.696743 16741 raft_consensus.cc:2410] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:19:12.589619 20003 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:19:12.700979 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:12.700120 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60149.1us
I20260709 13:19:12.702118 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:12.708470 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:12.720892 16666 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:19:12.683395 20559 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.727669 16591 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:19:12.731084 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:12.610308 20520 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
W20260709 13:19:12.734066 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54317.6us
I20260709 13:19:12.734376 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:12.622778 20372 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:19:12.693814 20396 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:12.737663 20559 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:12.722961 20550 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:12.719007 20028 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:12.725263 20551 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:12.769665 16739 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:12.819370 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.820298 16666 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:12.738173 20480 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:19:12.783138 20567 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:19:12.712571 20211 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.766835 20511 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:12.743366 20582 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:12.783466 20550 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.852197 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.784777 20240 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.860149 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:12.864742 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55430 (request call id 32) took 1132 ms. Trace:
I20260709 13:19:12.829027 20028 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:12.764559 20396 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.743140 20571 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.127s
I20260709 13:19:12.879153 16667 rpcz_store.cc:276] 0709 13:19:11.732255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.732399 (+   144us) service_pool.cc:224] Handling call
0709 13:19:12.864491 (+1132092us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54001b08d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbde853 00007f3284bac60a 00007f327e1327d4 00007f327e132762 00007f327e132691 00007f327e13260c 00007f327e1318c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:12.864615 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763920640}
I20260709 13:19:12.787185 20017 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:19:12.835706 20471 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.876194 16590 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:12.907831 16739 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:12.867645 20511 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:12.875393 20550 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:12.916417 20471 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.899237 20512 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.899775 20028 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.913293 20511 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:12.903083 20240 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.913726 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:12.909327 20546 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.918819 20471 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.959424 20026 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:12.920033 20480 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:12.992230 20480 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.966382 16742 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:12.970194 20557 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:12.973685 20394 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.978699 20559 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.979514 20545 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:12.982102 20581 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:19:12.987282 16593 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:12.988360 16743 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:12.922158 20525 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.923367 20579 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:12.931232 20407 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.939170 20017 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:12.947455 16666 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:13.047394 20017 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:12.951268 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:12.956373 20565 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.579s	user 0.000s	sys 0.002s
I20260709 13:19:13.048875 20017 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:12.956552 16589 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:12.994835 20480 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.945421 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.051806 17412 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:13.052266 16589 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:12.997015 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:12.999356 16492 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 219) took 33428 ms (client timeout 15000 ms). Trace:
I20260709 13:19:13.006945 20559 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.013830 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.028697 20521 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:13.055509 16490 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 5 to 6, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.057083 16743 raft_consensus.cc:2393] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:13.057214 20521 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.057984 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.059250 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.059990 16743 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:13.029021 16738 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:13.061908 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.062305 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.063848 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.064057 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:13.064811 16738 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:13.029348 16745 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:13.067526 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.068594 16741 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.069222 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.070083 16742 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:19:13.033718 17414 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:19:13.033851 20289 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.072717 20289 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:12.937800 20028 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.044858 16594 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:19:12.939579 20574 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.326s	user 0.000s	sys 0.001s
I20260709 13:19:12.938306 20211 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:12.939647 20483 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:12.941293 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.083035 16588 raft_consensus.cc:2410] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:19:13.002974 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.033245 20394 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.084555 20394 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.086813 20394 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:13.054616 16492 rpcz_store.cc:269] 0709 13:18:39.571010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:39.571242 (+   232us) service_pool.cc:224] Handling call
0709 13:19:12.999121 (+33427879us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:08.463490 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:08.523573 (+ 60083us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:08.523614 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:19:08.523631 (+    17us) tablet.cc:662] Decoding operations
0709 13:19:08.524715 (+  1084us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:08.524770 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:08.524799 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:19:08.525643 (+   844us) tablet.cc:701] Row locks acquired
0709 13:19:08.525666 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:19:08.525825 (+   159us) write_op.cc:270] Start()
0709 13:19:08.525944 (+   119us) write_op.cc:276] Timestamp: P: 1783603148525770 usec, L: 0
0709 13:19:08.525968 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:08.526726 (+   758us) log.cc:844] Serialized 4025 byte log entry
0709 13:19:09.074285 (+547559us) op_driver.cc:464] REPLICATION: finished
0709 13:19:09.445821 (+371536us) write_op.cc:301] APPLY: starting
0709 13:19:09.446004 (+   183us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:09.446339 (+   335us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:09.446490 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:09.822225 (+375735us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:09.912216 (+ 89991us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:19:09.916486 (+  4270us) write_op.cc:312] APPLY: finished
0709 13:19:10.051857 (+135371us) log.cc:844] Serialized 77 byte log entry
0709 13:19:10.095974 (+ 44117us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:10.131071 (+ 35097us) write_op.cc:454] Released schema lock
0709 13:19:10.149254 (+ 18183us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:10.159356 (+ 10102us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:10.159396 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3438696,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":363500,"apply.run_cpu_time_us":11475,"apply.run_wall_time_us":721126,"num_ops":10,"prepare.queue_time_us":127741,"prepare.run_cpu_time_us":13907,"prepare.run_wall_time_us":380565,"raft.queue_time_us":49161,"raft.run_cpu_time_us":8587,"raft.run_wall_time_us":289840,"replication_time_us":548115,"thread_start_us":507723,"threads_started":4,"wal-append.queue_time_us":253531,"wal-append.run_cpu_time_us":23422,"wal-append.run_wall_time_us":2384155}]]}
I20260709 13:19:13.086457 16490 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.092743 20555 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:13.093330 20555 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.093773 20555 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.095346 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:13.097054 20028 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:13.097697 20028 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:13.098838 20028 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.119267 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:13.121922 16710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.047s	sys 0.046s
W20260709 13:19:13.122401 16710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.047s	sys 0.046s
I20260709 13:19:13.124181 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.128424 20379 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:13.128894 20379 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:13.130012 20379 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.131839 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.132419 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.133292 16739 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:19:13.132525 17415 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:13.133102 16665 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:13.139588 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.157646 16490 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.159621 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.122220 20588 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:13.181450 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 115391us
I20260709 13:19:13.167428 16588 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:13.182255 16593 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:19:13.184576 16593 pending_rounds.cc:85] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 6
I20260709 13:19:13.184940 16593 pending_rounds.cc:107] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:13.185350 16593 raft_consensus.cc:2889] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:13.186798 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.195117 20582 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:13.196530 20582 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:13.196962 20582 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:13.197849 20582 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.181370 20546 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:19:13.141095 20218 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.199769 20546 consensus_queue.cc:1243] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:13.199967 20218 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.202296 20026 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.202847 20026 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.203063 20020 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:13.210767 16665 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:19:13.214731 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 174806us
I20260709 13:19:13.123224 20573 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 13:19:13.192628 20539 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:19:13.213833 20575 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:19:13.241530 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.242048 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.184041 20394 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.142611 20522 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.077s
W20260709 13:19:13.255541 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:13.144726 20423 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.006s	sys 0.011s
I20260709 13:19:13.267081 16490 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.206992 20588 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:13.243929 20603 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:13.217535 20545 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.269510 20573 consensus_queue.cc:1243] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:19:13.275681 20592 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:13.331688 20026 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.331248 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.278208 20533 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.332453 20511 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.333292 20533 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.280990 16739 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:13.324538 16741 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:19:13.264801 20394 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.325053 20545 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.332141 20586 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:13.269658 20471 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.351923 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:13.245582 20582 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:13.335915 16594 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:19:13.348789 16742 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:13.367708 17414 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.349520 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:13.268952 20423 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.007s	sys 0.011s
I20260709 13:19:13.353391 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.354322 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.358244 16665 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:13.361578 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:13.364928 16596 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:13.286183 20379 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:13.371520 17415 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:13.372925 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:13.377535 17420 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.373618 17416 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:13.334664 20483 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:13.334755 20511 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.358958 20559 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.386559 20559 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.386107 16490 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.387415 16741 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.388674 20559 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.380041 20501 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:19:13.359880 20586 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:13.393018 20586 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.394371 20586 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.395421 20550 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.396112 20533 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.397811 20545 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.397845 16587 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:13.400285 20550 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.362229 20289 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.374802 20471 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.403779 20471 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.404439 20555 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:13.405742 20531 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.000s
I20260709 13:19:13.406447 16588 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.406231 20531 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.421329 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.356567 20522 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:19:13.454800 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.459367 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.460518 16595 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:13.462263 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:13.461280 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.390337 16490 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:13.480413 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.480823 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.481798 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.376891 20028 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:19:13.481845 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.415357 20483 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.485865 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:13.488135 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.488884 16595 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:13.492863 20595 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:13.487149 20511 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:13.493306 20284 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:13.483209 20423 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:13.494089 20423 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.494454 20423 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.494949 20407 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:13.482941 20546 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:13.499012 20546 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.501209 20546 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:13.501724 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.509326 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:13.512625 16669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 401) took 1893 ms (client timeout 2165 ms). Trace:
W20260709 13:19:13.513090 16669 rpcz_store.cc:269] 0709 13:19:11.619322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:11.619452 (+   130us) service_pool.cc:224] Handling call
0709 13:19:13.512416 (+1892964us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54001f7054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:13.512539 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4105962240}
W20260709 13:19:13.514460 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 153840us
I20260709 13:19:13.516310 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:13.486299 20512 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:19:13.517589 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.518126 20512 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:13.518412 19911 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.518906 20512 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:19:13.485823 20522 consensus_queue.cc:1243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:19:13.510603 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.527594 16594 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:19:13.529774 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.530464 16745 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:13.484776 20480 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:13.486159 20593 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.520422 20407 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:13.539693 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.485369 20218 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.512625 20555 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.493314 20595 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.551246 16595 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:13.541316 20480 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.552302 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:13.558681 16745 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.555534 16596 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:19:13.548838 20545 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.530555 20511 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:13.569795 16595 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:19:13.571919 16595 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 5
I20260709 13:19:13.572191 16595 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:19:13.572459 16595 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:13.558128 20595 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:13.561756 20028 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.574950 20545 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.560597 20480 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.576413 20394 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:13.576838 20578 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.789s	user 0.006s	sys 0.003s
I20260709 13:19:13.593480 16745 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:13.573699 20423 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.580083 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:13.580449 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.623580 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 362) took 1095 ms. Trace:
I20260709 13:19:13.581073 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.626641 16592 rpcz_store.cc:276] 0709 13:19:12.527777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.668278 (+140501us) service_pool.cc:224] Handling call
0709 13:19:13.608943 (+940665us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b54003584d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:13.623473 (+ 14530us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845598592}
I20260709 13:19:13.625480 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.629837 17415 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:19:13.630137 16745 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:13.608363 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.632360 16587 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:19:13.628012 16669 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:13.634989 16595 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 5. Current term is 6. Ops: []
I20260709 13:19:13.608426 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:13.568908 20550 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:13.639477 20394 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:13.640993 20394 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.641381 20394 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.641667 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.642378 16738 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:13.643389 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:13.643982 16738 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.640336 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 526) took 1351 ms. Trace:
I20260709 13:19:13.644589 20483 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.644760 16737 rpcz_store.cc:276] 0709 13:19:12.288579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.288862 (+   283us) service_pool.cc:224] Handling call
0709 13:19:12.289224 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:12.289600 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:12.289633 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:12.290136 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:19:12.515073 (+224937us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:12.515121 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:12.515189 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:13.576246 (+1061057us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:13.639998 (+ 63752us) raft_consensus.cc:1685] finished
0709 13:19:13.640039 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:13.640252 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":179,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"spinlock_wait_cycles":12615936,"thread_start_us":224569,"threads_started":1,"wal-append.queue_time_us":354363}
I20260709 13:19:13.647066 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:13.650491 16744 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:13.573511 20492 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:19:13.622457 20026 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:13.611783 20512 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.659358 20512 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.659708 20510 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:13.660326 20510 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:13.661556 20512 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.662189 20240 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.662937 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:13.663568 16745 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:13.664907 17420 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:13.665781 16710 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.650272 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.666625 20547 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:13.666754 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.667057 20547 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.667428 20547 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:13.667394 16743 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:13.668536 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.668869 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.669119 16742 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:13.669378 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.669938 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.670075 20512 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:13.670490 20512 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.669885 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:13.670979 20512 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:13.676214 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.679344 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.679932 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.680634 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:13.680855 20519 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:19:13.681211 20519 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:13.680490 16490 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 7 to 8, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.682076 20519 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.686056 20519 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:13.684510 20559 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.687129 20559 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.687454 20521 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.689131 20026 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.689672 20026 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:13.690167 20607 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 13:19:13.684633 20549 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:13.693080 20549 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:13.694034 20549 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.695250 20549 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:13.689186 20559 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.699868 20396 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.700227 20423 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.701383 20531 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.707319 16737 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:13.685029 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.685360 20532 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.711902 16742 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:13.713570 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.716861 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.669927 17420 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
W20260709 13:19:13.712639 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.628863 20579 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.721510 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.728353 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.653937 20586 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.734723 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.736012 16744 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:13.736203 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:13.737308 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.737448 16744 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:13.739827 16744 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:13.712641 16738 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:13.727532 16742 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:13.741567 16742 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:13.731069 20532 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.706740 20204 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.742784 16741 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.746259 16595 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:13.750790 16593 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:13.653614 20483 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:13.754050 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.756129 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.764575 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:13.635612 20028 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.759457 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:13.767035 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:13.834546 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 352780us
I20260709 13:19:13.720808 20550 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:19:13.757843 20531 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:13.733240 20511 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:19:13.753544 20559 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:13.848695 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 89310.7us
I20260709 13:19:13.769277 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.860210 16593 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:13.854555 16745 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:13.757529 20582 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:13.741102 20573 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:13.860456 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:13.913889 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53570.5us
I20260709 13:19:13.915014 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.916500 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:13.918928 20573 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:13.765326 20483 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.921804 20483 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.866581 20511 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:13.922410 20511 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.922855 20511 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.870303 20017 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:13.866684 16739 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:13.862002 20379 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:13.932015 20379 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:13.872936 20423 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:13.875250 20396 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:13.876789 20519 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:13.859035 20471 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:13.934424 20471 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:13.934646 20519 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:13.935632 20519 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:13.937016 16744 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:13.937485 20545 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.883342 16490 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:13.938484 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:13.889635 20026 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.891523 20028 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.892841 20588 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:13.893642 20547 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.945115 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:13.945744 17415 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:13.946187 20547 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.896234 20593 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:13.896132 16742 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.750236 20586 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.947393 16742 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:13.947942 20480 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.948729 20026 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.948840 20586 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.881098 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:13.860639 20531 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:13.949638 20531 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.950043 20480 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:13.951304 20396 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:13.951865 20396 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:13.952754 20396 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:13.953131 19911 consensus_queue.cc:1175] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "170921ba08b74a33995ae7c7686f8109" responder_term: 11 status { last_received { term: 11 index: 7 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:19:13.954614 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 73641.4us
I20260709 13:19:13.958216 16742 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:13.903934 20510 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.959318 20510 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.907125 20394 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.919432 20559 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:13.858583 20204 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:19:13.919838 20616 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:13.963194 20616 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:13.934944 20471 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:13.936807 20519 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.971710 20519 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:13.886550 20532 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:13.745608 20579 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:13.977591 20579 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:13.981912 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:13.982942 16592 raft_consensus.cc:2368] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12.
I20260709 13:19:13.985666 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.986822 20204 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:19:13.988673 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 466525us
I20260709 13:19:13.995491 20492 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:13.995499 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:14.027284 17414 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:13.996416 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.000525 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.002122 16595 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:14.029589 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 111427us
I20260709 13:19:14.023021 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:14.033631 16594 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.021003 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2758) took 1350 ms. Trace:
I20260709 13:19:14.035693 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.036823 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.038376 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.040596 16662 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:14.035040 16736 rpcz_store.cc:276] 0709 13:19:12.670444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:12.978212 (+307768us) service_pool.cc:224] Handling call
0709 13:19:12.978675 (+   463us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:13.166241 (+187566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:13.166286 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:13.357851 (+191565us) log.cc:844] Serialized 52 byte log entry
0709 13:19:13.560753 (+202902us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:14.020166 (+459413us) log.cc:844] Serialized 14 byte log entry
0709 13:19:14.020549 (+   383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.020655 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:14.020713 (+    58us) raft_consensus.cc:1685] finished
0709 13:19:14.020730 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:14.020915 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136902,"prepare.queue_time_us":417243,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":158208,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":8768,"thread_start_us":525239,"threads_started":2,"wal-append.queue_time_us":349605}
I20260709 13:19:14.042068 16596 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:14.054260 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53826.7us
I20260709 13:19:14.055090 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:13.946439 20512 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:14.066499 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.074491 16559 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.042997 20284 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:19:14.082472 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.083263 16671 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.082742 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:14.083060 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56311.4us
I20260709 13:19:14.086810 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.097958 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.108831 16590 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:14.109935 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.096571 17414 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:14.113379 16594 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:13.943454 20595 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.028728 20624 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.112545 16664 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
W20260709 13:19:14.115734 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 387471us
W20260709 13:19:14.120627 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.124085 16634 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:14.129136 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:14.063341 20028 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:14.065610 20394 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:14.130280 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:14.087612 20545 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:14.065335 20579 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:14.143713 17412 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:14.144400 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.109139 20483 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.043184 20597 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 13:19:14.114673 20555 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:14.115058 20557 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:14.127947 20619 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:14.052714 20495 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:19:14.149639 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:14.132436 20629 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:14.152612 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.087675 20357 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:19:14.156948 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.115362 20240 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:19:14.051070 20524 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:19:14.146808 20423 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:14.143793 20545 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:14.159605 16664 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:14.157039 20547 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:14.084223 20531 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.085513 20549 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:19:14.161711 16634 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:14.166931 16662 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:14.119287 20406 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:19:14.138321 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:14.089591 20511 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.157783 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.219870 16589 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:14.043661 20492 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.222393 20492 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:14.150703 20394 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:14.179827 20471 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.181908 20595 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:14.168409 20512 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:14.167680 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 135533us
W20260709 13:19:14.185328 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.176865 20545 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:14.196473 20406 consensus_queue.cc:1243] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:14.196571 20240 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:14.197019 20204 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:14.199201 16671 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:14.183647 20547 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.143060 20635 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:19:14.236769 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.238426 16669 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:14.163892 20621 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:19:14.155946 20579 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.133993 20624 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.241122 20240 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.240890 20532 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:19:14.242712 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 104470us
W20260709 13:19:14.199913 20625 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:19:14.189244 19911 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:14.275322 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 188581us
I20260709 13:19:14.274502 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:14.276206 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.276961 20623 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:14.277505 20623 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:14.277715 20621 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:14.278103 20621 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:14.278102 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.278973 20621 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.280520 20621 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:14.271960 20284 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:19:14.276588 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:14.283936 20357 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:14.284356 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:14.284374 20480 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:14.285053 20513 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:14.241755 20204 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:14.286618 20204 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.287719 20204 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.290076 20616 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:14.279271 19911 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:14.292250 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.292388 19911 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:14.293205 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:14.295248 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.295004 20512 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:14.296041 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.276350 20240 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:14.278291 20623 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.235847 20559 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:14.301478 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.302227 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.303465 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.303731 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.307030 16669 pending_rounds.cc:85] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 6
I20260709 13:19:14.313069 16669 pending_rounds.cc:107] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:14.313418 16669 raft_consensus.cc:2889] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:14.248288 20126 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:19:14.342733 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64091.4us
I20260709 13:19:14.245704 20511 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:14.247222 20547 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:14.344990 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.249891 20521 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:14.279049 20184 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:19:14.349117 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55998.5us
I20260709 13:19:14.247501 20026 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:14.371274 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:14.371901 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.368518 16737 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:14.361675 16668 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.372376 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:14.276635 20597 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.249210 20396 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 13:19:14.381793 16745 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:14.341466 20211 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.341696 20532 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:14.322822 20555 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:14.346681 20126 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:14.376132 20406 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:14.311933 20480 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:14.378232 20284 consensus_queue.cc:1243] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:14.374850 20026 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:14.388763 20557 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:14.394827 20240 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.376164 20554 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:14.401494 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.467904 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:14.468586 17412 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:14.402469 20597 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:14.420104 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96520.7us
I20260709 13:19:14.420451 16668 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:19:14.421911 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 143865us
I20260709 13:19:14.430231 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.430449 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.431282 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.378839 20582 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.404073 20586 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:14.347885 20593 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:19:14.360837 20028 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:19:14.341549 20559 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.454113 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.476864 16664 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.465560 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:14.472199 16667 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:14.478139 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 132712us
I20260709 13:19:14.440902 20483 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:14.448280 20406 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.453675 20596 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:19:14.469619 20635 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:14.477129 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:14.484115 20635 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.484603 19911 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:14.485564 19911 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.485942 19911 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:14.466928 20026 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.432437 20211 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.489547 20211 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:14.492254 20240 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:14.493223 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.480043 16662 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:19:14.496904 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:14.497862 20559 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:14.498747 16667 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:14.499680 20184 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:14.500429 20184 consensus_queue.cc:1243] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:19:14.500360 20284 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:14.501037 20184 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:14.501411 20184 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:14.504232 16665 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:14.504323 20545 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.506371 20644 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:14.506230 19911 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.506778 20644 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:14.473420 20510 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.007s
I20260709 13:19:14.499899 20522 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:14.502251 20184 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.471048 20521 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.448777 20492 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
W20260709 13:19:14.518972 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.544768 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.545601 17409 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:14.547515 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.549048 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:14.549731 17420 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.523237 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.523627 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.474113 20555 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:14.538836 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.515153 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.565761 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.506829 20545 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:14.558928 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.516247 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:14.536926 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.477571 20020 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:14.537957 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.571259 16737 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:14.524209 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.527706 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:14.524462 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.572698 17416 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:14.516749 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.525108 16664 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:14.523896 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.525076 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:14.528404 20184 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.516413 20017 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:19:14.507658 20644 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.557003 20521 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:14.575153 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.592658 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:14.571269 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.593297 16667 raft_consensus.cc:606] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.593791 16667 raft_consensus.cc:614] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:14.569310 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53160.3us
I20260709 13:19:14.574543 16740 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.569792 16742 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:14.582643 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.582804 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:14.593931 16666 raft_consensus.cc:2410] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 10 because replica has last-logged OpId of term: 9 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:19:14.576012 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.595086 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:14.474895 20554 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.598173 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.508549 19911 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:14.590071 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92370.1us
I20260709 13:19:14.510954 20531 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:14.598228 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 274010us
I20260709 13:19:14.607249 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:14.625239 17420 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:14.636308 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:14.636777 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:14.658962 16589 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:14.639192 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.635847 16662 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 11. (term mismatch)
I20260709 13:19:14.659888 16662 pending_rounds.cc:85] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 10
I20260709 13:19:14.659943 16594 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:14.660250 16662 pending_rounds.cc:107] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 9.11
I20260709 13:19:14.660542 16662 raft_consensus.cc:2889] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:14.661403 16594 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:14.525522 20586 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:14.644019 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.649654 16595 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:19:14.664361 16741 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:19:14.649454 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.665268 16738 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:19:14.666240 16664 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:14.511233 20512 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:19:14.512344 20522 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:14.651605 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:14.720489 16596 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.664830 16741 pending_rounds.cc:85] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:19:14.721459 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.567978 20635 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:14.479269 20582 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.666265 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.650801 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:14.725878 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.726069 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.727814 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.674286 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.585239 20519 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:14.729336 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.730468 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.579180 20549 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:14.731609 20549 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:14.577090 20593 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.580099 20233 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:14.705065 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:14.610882 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.209s (SENT) [suppressed 7 similar messages]
I20260709 13:19:14.707242 16738 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:14.573652 20650 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.580626 20471 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:14.576979 20211 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.580374 20579 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:14.577194 20483 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:14.721688 16741 pending_rounds.cc:107] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:19:14.700282 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:14.574546 20588 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:14.590925 20642 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:14.732419 16670 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:14.732353 16743 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:14.585860 20624 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:14.734720 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:14.733405 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.738205 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:14.832887 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 98317.5us
I20260709 13:19:14.646219 20545 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:14.531070 20126 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.740831 20593 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.592597 20020 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.531522 20240 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.204s
I20260709 13:19:14.647374 20632 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:14.747869 20597 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:14.654166 20017 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.647025 20559 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:14.735541 16591 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:14.576077 20204 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.164s
I20260709 13:19:14.509127 20480 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:14.727249 20582 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:14.583613 20555 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:14.821725 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 214452us
I20260709 13:19:14.830503 16741 raft_consensus.cc:2889] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:14.750119 16742 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:14.784718 16737 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:19:14.836086 16596 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:14.928321 16738 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 13:19:14.751945 17416 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
W20260709 13:19:14.930266 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.930734 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:14.732898 20495 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:14.730939 20557 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:14.966151 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:14.984313 16741 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:14.985991 16741 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:19:14.992466 16594 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:14.736290 19911 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:14.783352 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:14.932157 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 295904us
W20260709 13:19:14.833966 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 223139us
W20260709 13:19:14.930738 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:15.000536 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 217313us
I20260709 13:19:14.781339 20595 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:14.749574 20531 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:14.996553 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:14.998203 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:15.009032 17415 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:15.001494 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:15.003357 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:14.861016 20554 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:14.830940 20642 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:15.014720 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:14.767225 20646 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:14.987709 20495 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:14.803171 16740 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:14.852005 20648 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:14.927685 20545 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.019124 16708 leader_election.cc:400] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12.
I20260709 13:19:14.941334 20559 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:14.749086 20233 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:15.026047 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:14.993304 20547 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.002s
I20260709 13:19:15.065519 16736 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:14.986378 20471 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:14.831964 20211 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:15.048622 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:14.983246 20017 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.035000 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:14.856345 20483 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.079284 17410 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:15.067768 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:15.023026 20510 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:19:15.044552 16740 raft_consensus.cc:1217] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 6.4->[8.5-11.7]   Dedup: 8.5->[9.6-11.7]
I20260709 13:19:15.083956 16740 pending_rounds.cc:85] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 5
I20260709 13:19:15.084373 16740 pending_rounds.cc:107] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:19:15.084682 16740 raft_consensus.cc:2889] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:15.084944 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:15.085968 20579 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:15.038993 20394 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:14.837173 20549 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.021252 20623 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:14.856523 20650 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.066442 20545 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:15.020836 19911 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.028594 20597 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:19:15.045284 20554 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:19:15.119083 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:19:15.140618 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 121189us
W20260709 13:19:15.119369 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.043367 20657 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:15.066188 20559 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:14.992646 20522 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.022151 20595 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:14.857223 20586 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.151360 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:15.162278 16671 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:15.098989 20471 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:15.163476 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.124426 20616 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:15.155115 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:15.197244 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:15.207525 20673 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:19:15.209640 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:15.131572 20406 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:15.233808 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.244565 17410 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:15.246455 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:15.087699 20547 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.154247 20657 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:15.168924 20026 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.004s
I20260709 13:19:15.198804 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:15.158609 20588 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:15.292986 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 46624.9us
I20260709 13:19:15.132472 20623 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:15.340626 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:15.355947 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 157228us
I20260709 13:19:15.340759 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:15.358165 16668 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:15.279058 20616 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:15.289196 20573 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:15.133991 20495 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.362720 20683 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:15.158735 20233 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:15.370455 16589 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.702s	user 0.005s	sys 0.002s
I20260709 13:19:15.374953 16589 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
W20260709 13:19:15.376459 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.286721 20657 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:15.382797 20650 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:15.385041 20547 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:15.385718 20547 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:15.292964 20020 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:15.158332 20557 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.387050 20573 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.387213 20020 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:15.387552 20573 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.387893 20020 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.389725 20573 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:15.390364 20020 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:15.386233 20547 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.363770 20582 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:15.290966 20026 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.354566 20648 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:15.382426 20017 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:15.378120 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:15.393667 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.359467 20640 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:15.374943 20658 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.088s	user 0.006s	sys 0.000s
W20260709 13:19:15.358402 20184 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.781s	user 0.006s	sys 0.010s
I20260709 13:19:15.361335 20593 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:15.444833 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.396000 20495 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:15.372651 20588 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:19:15.417444 20233 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:15.479458 20685 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:15.480304 20681 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:15.618539 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:15.619313 16736 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:15.619496 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:15.491164 20687 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 13:19:15.492301 20677 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
W20260709 13:19:15.493577 20662 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 13:19:15.497608 20648 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:15.498952 20659 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.142s	user 0.000s	sys 0.002s
I20260709 13:19:15.392443 20683 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.420939 20582 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:15.468606 20640 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.518080 20642 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.372903 20471 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.602597 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 407) took 1085 ms. Trace:
I20260709 13:19:15.663674 20028 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.603485 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.603837 20211 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:15.605013 16559 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:19:15.606820 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.690944 16670 rpcz_store.cc:276] 0709 13:19:14.517498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.517613 (+   115us) service_pool.cc:224] Handling call
0709 13:19:14.573939 (+ 56326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:14.831017 (+257078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.525111 (+694094us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400346f54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:15.602401 (+ 77290us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b5400346f50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:15.602534 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492756480}
I20260709 13:19:15.663928 19911 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:15.607759 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2786) took 1080 ms. Trace:
I20260709 13:19:15.608438 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:15.672415 20667 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:15.613152 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 2779) took 1090 ms. Trace:
I20260709 13:19:15.615051 16737 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:19:15.488682 20663 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:15.620450 17416 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:15.685410 20595 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:15.667948 20666 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:15.668459 20660 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.545s	user 0.000s	sys 0.005s
I20260709 13:19:15.618762 20644 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:15.640657 20621 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.360s	user 0.008s	sys 0.008s
I20260709 13:19:15.619618 20632 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:15.649435 20593 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.651475 20638 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:15.741024 16666 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.012s	sys 0.017s
W20260709 13:19:15.694160 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.701979 20682 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:15.469497 20669 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:15.694110 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 89229.2us
I20260709 13:19:15.692696 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:15.698786 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90422.6us
I20260709 13:19:15.754402 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:15.747781 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.755267 17415 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:15.745716 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.746236 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.611645 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:15.697324 16738 rpcz_store.cc:276] 0709 13:19:14.527523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.751041 (+223518us) service_pool.cc:224] Handling call
0709 13:19:14.928038 (+176997us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:14.928834 (+   796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.554670 (+625836us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540005a850. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:15.607695 (+ 53025us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406125824}
I20260709 13:19:15.701843 16743 rpcz_store.cc:276] 0709 13:19:14.522448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.582371 (+ 59923us) service_pool.cc:224] Handling call
0709 13:19:15.523626 (+941255us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400049554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:15.613089 (+ 89463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313904000}
W20260709 13:19:15.620707 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.667609 20483 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:15.621932 20685 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.730094 17416 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:15.696342 20623 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:15.678776 20616 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:15.768371 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:15.753147 20674 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.502s	user 0.004s	sys 0.000s
I20260709 13:19:15.722337 20642 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:15.771880 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:15.772624 17415 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:15.761049 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:15.761325 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 149791us
I20260709 13:19:15.762050 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:15.699481 20028 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.775532 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.764715 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.766676 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:15.766625 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 74012us
I20260709 13:19:15.753530 20026 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:15.782639 16668 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 5.
I20260709 13:19:15.783706 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2792) took 1211 ms. Trace:
I20260709 13:19:15.784263 16740 rpcz_store.cc:276] 0709 13:19:14.572511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.043961 (+471450us) service_pool.cc:224] Handling call
0709 13:19:15.044338 (+   377us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:15.085524 (+ 41186us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:19:15.085920 (+   396us) log.cc:844] Serialized 14 byte log entry
0709 13:19:15.118737 (+ 32817us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:15.503481 (+384744us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b2c00141498. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:19:15.504092 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 13:19:15.504256 (+   164us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:15.504294 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.504372 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.651063 (+146691us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b180015aa20. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f
0709 13:19:15.668995 (+ 17932us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b180015aa20. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7522 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:15.783514 (+114519us) raft_consensus.cc:1685] finished
0709 13:19:15.783536 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.783649 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128816,"prepare.queue_time_us":10004,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":5945,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10398,"spinlock_wait_cycles":63282432,"thread_start_us":31564,"threads_started":1,"wal-append.queue_time_us":204695}
I20260709 13:19:15.788899 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:15.789104 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.779456 16634 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:15.701577 20667 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.502136 20657 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.753934 20547 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:15.767102 16666 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.013s	sys 0.018s
I20260709 13:19:15.793542 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:15.810390 16560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.007s	sys 0.010s
I20260709 13:19:15.769013 16665 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:15.792021 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.007s	sys 0.010s
I20260709 13:19:15.788612 20645 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:19:15.766310 20621 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:15.793116 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:15.771111 20664 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:15.817178 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:15.811735 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:15.817055 16560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.010s
W20260709 13:19:15.817751 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.007s	sys 0.010s
W20260709 13:19:15.829602 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 803627us
I20260709 13:19:15.745270 20593 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.824293 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:15.724483 20663 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.824146 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 437) took 1251 ms. Trace:
W20260709 13:19:15.824385 20624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.004s	sys 0.014s
I20260709 13:19:15.782944 20595 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:15.857263 16665 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:15.792177 20531 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:15.840308 17412 rpcz_store.cc:276] 0709 13:19:14.542445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.542603 (+   158us) service_pool.cc:224] Handling call
0709 13:19:14.542948 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:14.543311 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:14.543338 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:14.543776 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 13:19:14.719656 (+175880us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:14.719713 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:14.719812 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:15.516204 (+796392us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:15.783311 (+267107us) raft_consensus.cc:1685] finished
0709 13:19:15.783329 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:15.793908 (+ 10579us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":6683520,"thread_start_us":175421,"threads_started":1,"wal-append.queue_time_us":458624}
I20260709 13:19:15.754134 20638 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:15.908000 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 77630.4us
I20260709 13:19:15.742384 20632 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:15.831188 20664 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:15.925498 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 146122us
I20260709 13:19:15.763936 20582 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:15.927253 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 103088us
W20260709 13:19:15.902388 20624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.005s	sys 0.014s
I20260709 13:19:15.906126 20550 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:15.928751 20631 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:15.736721 20666 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:15.932457 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 170483us
W20260709 13:19:15.931460 20646 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.888s	user 0.008s	sys 0.003s
W20260709 13:19:15.920595 20521 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.323s	user 0.013s	sys 0.001s
I20260709 13:19:15.938011 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:15.929814 16663 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
W20260709 13:19:15.924181 20545 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.783s	user 0.006s	sys 0.009s
I20260709 13:19:15.941455 17415 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:15.920040 20595 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:15.837983 20204 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 13:19:15.789878 20681 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.148s
I20260709 13:19:15.813555 20680 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.157s
I20260709 13:19:15.930006 20664 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:15.937024 20524 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:15.937163 20512 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:15.982836 16667 raft_consensus.cc:606] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:15.986536 20646 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:16.001042 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:15.961266 20692 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:15.966008 20555 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:15.939854 20550 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:16.020679 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 82780.2us
I20260709 13:19:16.033651 17412 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [7.7-8.8]
I20260709 13:19:16.045377 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:16.047538 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:15.933243 20616 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.066509 16671 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:15.989771 20483 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.011559 20595 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:15.998601 20698 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:16.084961 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:16.019646 20524 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:15.942442 20631 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:15.985982 20521 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:16.098419 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.035872 20512 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.095191 20675 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.066293 20218 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:16.065696 20396 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:15.991253 20545 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:16.097221 20495 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.624s	user 0.008s	sys 0.002s
I20260709 13:19:16.093828 20211 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:16.101577 20524 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:16.093340 20555 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.033674 20623 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:16.115252 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:16.114666 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:16.160303 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 45138us
I20260709 13:19:16.097412 20698 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:16.160427 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45831.7us
I20260709 13:19:16.103341 20483 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.132486 20593 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.179572 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:16.139750 20211 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:16.139593 20480 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.175531 20693 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:16.193233 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60001.7us
I20260709 13:19:16.133692 20218 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.166222 20020 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:16.109215 20675 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.132090 20406 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:19:16.187937 20559 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.968s	user 0.012s	sys 0.000s
I20260709 13:19:16.120234 20396 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.182446 20623 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:16.199187 20705 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:19:16.184274 20701 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.204879 20020 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.193953 20683 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.203969 20657 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.244820 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:16.205463 20593 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:16.207614 20685 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.209950 20638 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.199574 20480 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:16.247815 20704 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:19:16.240928 20028 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.212966 20667 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:16.223011 20680 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:16.289633 20020 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.208788 20218 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.212193 20696 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.298s
I20260709 13:19:16.296186 20683 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:16.321347 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1337) took 1568 ms. Trace:
W20260709 13:19:16.312150 20715 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 13:19:16.331718 16738 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:16.333181 20710 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:19:16.320453 20694 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:19:16.333945 20714 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:16.335701 20480 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:16.336719 20699 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:19:16.362910 16589 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:16.285815 20701 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.292270 20698 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.343793 16670 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:19:16.344655 20707 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 13:19:16.346030 20483 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.222350 20559 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.348944 20396 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.306936 20593 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.390443 20666 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:16.350246 20233 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.000s
I20260709 13:19:16.350636 20218 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.326625 16745 rpcz_store.cc:276] 0709 13:19:14.752503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.069299 (+316796us) service_pool.cc:224] Handling call
0709 13:19:15.118126 (+ 48827us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:15.119827 (+  1701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:15.119883 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:15.490805 (+370922us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b18001bd720. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a
0709 13:19:15.491263 (+   458us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b2c00141498. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:19:15.491970 (+   707us) log.cc:844] Serialized 52 byte log entry
0709 13:19:15.766900 (+274930us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:15.801612 (+ 34712us) log.cc:844] Serialized 14 byte log entry
0709 13:19:16.101518 (+299906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:16.101590 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.321016 (+219426us) raft_consensus.cc:1685] finished
0709 13:19:16.321046 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.321267 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15832,"prepare.queue_time_us":35029,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":7622,"raft.queue_time_us":106084,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":69046,"spinlock_wait_cycles":26282624,"thread_start_us":574111,"threads_started":2,"wal-append.queue_time_us":540384}
W20260709 13:19:16.333552 20706 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 13:19:16.391757 20697 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 13:19:16.360983 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.316812 20545 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.336589 20675 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.365233 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:16.365913 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.366526 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.395252 16587 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.367395 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.367514 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.289551 20211 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.396333 16596 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:16.395457 16591 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:16.370114 16590 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:19:16.397444 16592 raft_consensus.cc:2368] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:16.395321 16593 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:16.398947 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.370766 20698 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:16.372233 20126 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:16.374246 20621 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:16.401589 20621 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:16.402415 20621 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.356734 20683 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.375674 20549 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:16.378152 20701 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.381224 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55430 (request call id 34) took 1040 ms. Trace:
I20260709 13:19:16.384084 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.384810 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.406661 16589 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 6.
I20260709 13:19:16.320113 20623 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.365998 20714 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.408489 20683 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.388255 20555 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.409694 16588 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:16.409838 20714 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:16.415940 20644 consensus_meta.cc:287] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.679s	user 0.010s	sys 0.002s
I20260709 13:19:16.391134 16741 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:16.392937 16594 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:16.393317 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.368263 20631 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.419502 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.420190 16590 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:16.400380 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1535) took 1250 ms. Trace:
I20260709 13:19:16.390759 20593 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.405718 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:16.406424 16667 rpcz_store.cc:276] 0709 13:19:15.340509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.340628 (+   119us) service_pool.cc:224] Handling call
0709 13:19:16.377087 (+1036459us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b540018f954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbde853 00007f3284bac60a 00007f327e1327d4 00007f327e132762 00007f327e132691 00007f327e13260c 00007f327e1318c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:16.381138 (+  4051us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436881664}
I20260709 13:19:16.387065 20559 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:16.421231 16669 rpcz_store.cc:276] 0709 13:19:15.150405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.150542 (+   137us) service_pool.cc:224] Handling call
0709 13:19:15.150910 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:15.281007 (+130097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:15.281050 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:15.281642 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:19:15.620791 (+339149us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:15.620860 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:15.621436 (+   576us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:16.400209 (+778773us) raft_consensus.cc:1685] finished
0709 13:19:16.400229 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:16.400331 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":129682,"raft.queue_time_us":309497,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":9055,"thread_start_us":248171,"threads_started":1,"wal-append.queue_time_us":573547}
I20260709 13:19:16.431048 16587 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:16.432508 20020 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:16.453552 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 47920.6us
I20260709 13:19:16.460150 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:16.569991 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.605024 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:16.606027 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:16.607025 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:16.607323 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:16.607897 16594 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.639703 16595 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:16.659772 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:16.664357 16708 leader_election.cc:400] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:16.665774 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.666241 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:16.461804 20593 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:16.672438 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:16.674072 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:16.664419 20588 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:16.665520 20616 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:16.664280 20555 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.666813 20020 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:16.681104 20233 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:16.663826 20211 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.681885 20635 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:16.681934 20586 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:16.692961 20588 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.692660 20616 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.724145 20593 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:16.727901 20664 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.000s
I20260709 13:19:16.724088 20233 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.727644 20635 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.727308 20545 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:16.698601 16594 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:16.728053 20586 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.747784 20709 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:16.747799 20595 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.644s	user 0.008s	sys 0.003s
I20260709 13:19:16.734679 20588 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.747541 20616 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:16.734064 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60085.3us
I20260709 13:19:16.754714 20670 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.763859 20635 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:16.766165 20233 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:16.765157 20586 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:16.768164 20621 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:16.729223 20211 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.773435 20709 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:16.726119 20555 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.763346 20664 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.768465 20545 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:16.819829 20724 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.360s	user 0.000s	sys 0.002s
W20260709 13:19:16.818761 20642 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.040s	user 0.005s	sys 0.014s
I20260709 13:19:16.780296 20670 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.773800 20595 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.842358 20396 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.844861 20593 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.817199 20020 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:16.844290 20656 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:16.852087 16670 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:19:16.816960 20675 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.821225 20722 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:16.847146 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:16.858839 20709 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.845800 20720 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:16.899379 20720 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.901662 20720 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:16.858240 20728 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:16.835902 20678 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.907213 20678 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.909750 20678 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.911880 20664 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.913818 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:16.914587 16588 raft_consensus.cc:2368] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
W20260709 13:19:16.917990 20698 consensus_meta.cc:287] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.000s
I20260709 13:19:16.920604 20545 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.860260 20211 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.870106 16662 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:19:16.922495 16662 pending_rounds.cc:85] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 6
I20260709 13:19:16.922741 16662 pending_rounds.cc:107] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:19:16.923009 16662 raft_consensus.cc:2889] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.923321 16662 pending_rounds.cc:107] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:19:16.923539 16662 raft_consensus.cc:2889] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:16.924146 20549 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.924655 20549 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:16.875530 20726 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:19:16.880118 20396 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.929327 20396 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.881151 20020 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.884682 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:16.881026 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 789) took 1090 ms. Trace:
W20260709 13:19:16.929347 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:16.872427 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:16.888556 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:16.932847 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:16.933126 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:16.931279 17415 rpcz_store.cc:276] 0709 13:19:15.790565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.792890 (+  2325us) service_pool.cc:224] Handling call
0709 13:19:16.880844 (+1087954us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540050b754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:16.880946 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272485760}
I20260709 13:19:16.933943 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:16.934834 16669 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:16.936568 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:16.939599 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:16.861543 20524 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.001s
I20260709 13:19:16.944437 17414 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:16.887632 20722 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.933158 20559 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.951256 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:16.885768 20670 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.938798 20644 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.940841 20218 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:16.963842 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:16.885120 20588 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.887446 20675 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.972224 20020 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:16.951896 20722 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:16.975980 20595 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:16.977268 20675 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.947654 20524 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.980633 16590 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:16.981775 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:16.983062 20020 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:16.982510 20621 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:19:16.883428 20656 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:16.963327 20644 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.956992 20666 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:16.959661 20218 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:16.980676 20524 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:16.960865 20559 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:16.994067 20218 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:16.978508 20635 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.021685 16593 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:16.993122 20666 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.040221 20586 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.041103 20666 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.043035 20644 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.042898 20233 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.045104 20233 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:17.052567 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 119487us
I20260709 13:19:17.055177 16593 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:17.066519 16595 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:17.069976 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:17.070930 16671 raft_consensus.cc:606] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:17.071376 16671 raft_consensus.cc:614] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:17.070946 20521 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.073026 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 2158) took 1328 ms. Trace:
I20260709 13:19:17.073400 16663 rpcz_store.cc:276] 0709 13:19:15.744343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.763244 (+ 18901us) service_pool.cc:224] Handling call
0709 13:19:17.072846 (+1309602us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540009cc54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:17.072968 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350086016}
I20260709 13:19:17.074199 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:17.076340 20512 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.078853 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.079255 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.080294 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.082499 20737 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.083148 20737 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.083501 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.084769 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.085546 20737 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.086242 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.086444 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.060894 20233 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.060997 20738 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.090819 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.101728 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.103315 20738 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:17.104768 20695 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.898s	user 0.000s	sys 0.010s
I20260709 13:19:17.105731 20738 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.107245 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:17.107414 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.111866 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:17.112545 16595 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:17.114652 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.121862 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:17.121979 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:17.114569 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.089872 20737 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:17.124929 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.125837 16742 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:17.112167 20738 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.133236 16671 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:17.133092 16663 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:19:17.134527 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:17.137686 16663 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 5
I20260709 13:19:17.111891 20586 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.149343 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.153586 16595 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:17.156744 20656 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.148455 20743 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.157672 20743 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.159235 20549 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.159262 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:17.161101 20663 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.132280 20616 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.163326 20640 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.164989 20616 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.165863 20616 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.122831 20735 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.167145 16739 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:17.107976 20233 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:17.166523 16668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 2156) took 1422 ms (client timeout 1704 ms). Trace:
W20260709 13:19:17.168073 16744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1331) took 2647 ms (client timeout 3156 ms). Trace:
I20260709 13:19:17.138015 16663 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 11.6
W20260709 13:19:17.189409 16744 rpcz_store.cc:269] 0709 13:19:14.520366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.523433 (+  3067us) service_pool.cc:224] Handling call
0709 13:19:17.167877 (+2644444us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b5400434254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:17.167990 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4853632896}
I20260709 13:19:17.190291 16663 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:17.189167 16668 rpcz_store.cc:269] 0709 13:19:15.743725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.747498 (+  3773us) service_pool.cc:224] Handling call
0709 13:19:17.166349 (+1418851us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b5400346a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:17.166465 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62638976}
I20260709 13:19:17.167532 20735 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.131148 20555 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.194646 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.187253 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:17.195294 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.196281 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.197650 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.208015 16595 raft_consensus.cc:2368] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7.
I20260709 13:19:17.208950 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.154670 20680 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:17.210659 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:17.212121 16594 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:17.213627 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.213876 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.213629 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.132038 20737 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.191179 20588 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.191425 20642 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:17.225082 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.207362 20512 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.146891 20218 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.191989 20623 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.214578 20680 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:17.221280 20666 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:19:17.227003 20733 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:19:17.205097 20586 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:17.145164 20738 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.230883 16589 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:17.191819 20616 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.203510 20394 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:19:17.228510 20642 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.242409 16587 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:17.239598 20588 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.243443 20588 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.246019 20588 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.237166 20640 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:17.248355 20218 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.248757 20635 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.238953 20623 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.249562 20586 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:17.250452 20586 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.251116 20623 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.251670 16596 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:17.254056 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.256683 16663 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:17.259747 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:17.260589 20663 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 13:19:17.273226 20744 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:19:17.274816 20640 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:17.275269 20640 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:17.276228 20640 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.277529 20640 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.279141 20623 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:17.279613 20623 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.280009 20623 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:17.284854 20616 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.242551 20642 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.292871 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:17.293526 16670 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:17.294929 20734 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.278s	user 0.000s	sys 0.002s
I20260709 13:19:17.297016 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 640) took 1525 ms. Trace:
I20260709 13:19:17.297559 17414 rpcz_store.cc:276] 0709 13:19:15.771805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.771963 (+   158us) service_pool.cc:224] Handling call
0709 13:19:17.296794 (+1524831us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400571054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:17.296941 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2655807360}
I20260709 13:19:17.235065 20512 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.311704 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1541) took 2152 ms. Trace:
W20260709 13:19:17.312117 20736 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:19:17.312242 16662 rpcz_store.cc:276] 0709 13:19:15.159249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.159436 (+   187us) service_pool.cc:224] Handling call
0709 13:19:15.159765 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.923997 (+1764232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:17.311359 (+387362us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54003df4d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:17.311580 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3932436352}
W20260709 13:19:17.315696 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 193809us
I20260709 13:19:17.317266 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:17.258965 20675 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:19:17.323746 16708 leader_election.cc:400] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:17.324180 16708 leader_election.cc:403] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:19:17.324796 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 69880.5us
I20260709 13:19:17.335170 16670 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:17.335909 16634 leader_election.cc:400] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7.
I20260709 13:19:17.336228 16634 leader_election.cc:403] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:17.336349 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:17.338672 20026 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:17.327699 20743 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.340906 20026 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:17.319211 20511 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:19:17.326186 20579 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.329597 20720 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:17.330221 20586 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.333106 20623 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.331037 20675 consensus_queue.cc:1243] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:17.351837 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:17.348531 20579 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.348142 20233 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.351935 20720 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.341898 20026 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:17.353818 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.355792 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:17.469851 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 114119us
I20260709 13:19:17.472182 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:17.413264 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:17.479615 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.486631 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:17.344302 20755 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:17.381580 20752 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:19:17.346053 20758 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:19:17.382162 20737 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.379347 20743 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.529521 20743 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:17.530421 20743 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:17.532889 20554 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:17.528669 20616 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:17.381489 20738 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.391558 20512 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.389660 20720 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.542194 20666 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:17.539973 20722 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.000s
I20260709 13:19:17.533674 20554 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.392479 20396 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:17.384985 20579 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.471065 20678 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.502060 20755 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:17.353227 20760 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:17.489476 20635 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:17.666684 16663 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:19:17.352306 20586 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.558708 20666 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.674139 20666 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.676224 20554 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.602118 20396 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:17.667475 16663 pending_rounds.cc:85] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 6
I20260709 13:19:17.668329 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.537489 20680 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:17.635027 17374 connection.cc:256] RPC call timeout handler was delayed by -1.5225862420002159s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 13:19:17.683748 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.568240 20722 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.681341 16663 pending_rounds.cc:107] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:19:17.683298 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.666832 20755 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.692863 16744 raft_consensus.cc:2368] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7.
W20260709 13:19:17.684158 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.694517 20554 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.684679 16743 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:17.583729 20737 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.680966 16591 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
W20260709 13:19:17.697011 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 202967us
I20260709 13:19:17.662559 20738 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.692379 16663 raft_consensus.cc:2889] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.698489 16743 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:17.668874 20678 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.701907 16592 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:19:17.702754 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.703975 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:17.705904 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.704921 20394 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:19:17.696884 20554 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.705964 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:17.710835 20554 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.708011 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:17.715586 20554 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:17.716005 16671 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.716439 20758 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 6. Current term is 7.
I20260709 13:19:17.716130 20554 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.717002 20735 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.718521 20554 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.719336 20554 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:17.719807 20754 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.391s	user 0.007s	sys 0.005s
I20260709 13:19:17.708856 16745 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 2 to 5, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:17.719851 20554 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:17.721429 20727 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.957s	user 0.011s	sys 0.019s
I20260709 13:19:17.722055 20755 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:17.722707 20554 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.709067 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.723711 16591 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:17.724545 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.725589 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.726481 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:17.730666 16591 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:17.730949 20532 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.731590 20532 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.733767 20532 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:17.706456 16664 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
W20260709 13:19:17.709589 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:17.740447 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:17.745826 16668 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:19:17.746297 16668 pending_rounds.cc:85] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 6
I20260709 13:19:17.746565 16668 pending_rounds.cc:107] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:19:17.746848 16668 raft_consensus.cc:2889] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:17.752240 16671 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:17.754328 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.756570 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:17.738442 20757 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:19:17.771507 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.773165 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:17.778630 20764 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:17.774060 16664 raft_consensus.cc:2368] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:17.700892 20738 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.699687 20737 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.713110 20760 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:17.803762 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:17.810576 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:17.811201 17416 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:17.812062 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.812173 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.812453 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.812675 16743 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:17.813061 16737 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:17.813798 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:17.814239 16591 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:17.814646 17416 raft_consensus.cc:2410] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:17.815919 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:17.816219 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.816717 17416 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:17.816689 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.816890 16736 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:17.817103 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.818178 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.809199 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:17.745337 20675 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.818178 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:17.818435 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.819161 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.819612 16589 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:17.819824 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:17.820753 16667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.007s
W20260709 13:19:17.822813 16667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.006s	sys 0.007s
I20260709 13:19:17.793901 16663 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:17.820672 16591 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:17.823278 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:17.809095 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59532.2us
I20260709 13:19:17.757468 20758 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:17.874948 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:17.798834 20759 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:19:17.801209 20722 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.807489 20678 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.885116 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.882170 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.881817 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:17.930773 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:17.943001 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:17.946040 17412 raft_consensus.cc:2410] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:17.933522 16743 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:17.923574 16736 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:17.930039 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52331.8us
I20260709 13:19:17.948990 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:17.949129 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.949786 16743 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:17.943715 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.951746 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.943996 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:17.946218 17420 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:17.953064 16591 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:17.953977 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.954094 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:17.945657 16737 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:17.990801 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.955227 16664 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:17.924324 20763 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.000041 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.800529 20755 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:17.745440 20396 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:17.961392 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:17.947713 20623 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:17.989890 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:17.992882 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.013203 16736 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:17.993355 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.821310 20554 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:17.994298 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:17.999070 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50153.8us
W20260709 13:19:17.999382 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.020323 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.020917 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.021736 16740 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:17.802240 20680 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.000876 16743 raft_consensus.cc:2368] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:18.002280 20737 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.002913 20586 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:18.003294 20642 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.000s
I20260709 13:19:18.005755 20738 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.006188 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2788) took 3473 ms. Trace:
I20260709 13:19:18.007405 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.934131 20758 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.011328 16591 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:18.048132 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.014674 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:17.875000 20675 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.051479 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.047050 16741 rpcz_store.cc:276] 0709 13:19:14.533099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:14.987225 (+454126us) service_pool.cc:224] Handling call
0709 13:19:14.987527 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:16.417619 (+1430092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.005907 (+1588288us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540005bc54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:18.006104 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3227428736}
I20260709 13:19:18.045940 20642 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.033946 16587 raft_consensus.cc:1217] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Deduplicated request from leader. Original: 7.7->[8.8-8.8]   Dedup: 8.8->[]
I20260709 13:19:18.016572 20760 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:18.053072 16671 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:18.053220 16587 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:19:18.053254 20760 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:18.053857 20760 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.054365 20642 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.036127 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:18.058655 16588 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.012089 20763 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.059473 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.060860 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.025470 17376 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:18.062175 17376 leader_election.cc:403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:19:18.032780 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.024559 16589 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:18.044548 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.065881 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.066568 16671 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:17.874637 20764 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:17.821998 20357 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.195s
I20260709 13:19:18.073300 20764 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.049801 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.076886 16591 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:18.034101 20554 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.052623 20675 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.026609 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.080916 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.080905 20554 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.064139 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.082954 16664 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:18.083884 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.084810 16664 raft_consensus.cc:2410] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 15 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:19:18.085805 16664 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.048924 16743 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:18.024708 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.098771 20738 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.024185 20770 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:18.101529 16587 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:18.102164 16744 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.102977 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:18.104336 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79010.6us
I20260709 13:19:18.109647 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.110518 16744 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.115769 16737 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:18.120009 16588 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:18.121058 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.045631 20396 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:18.125721 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:18.100232 20752 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:19:18.146549 20675 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.154366 20675 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.156517 20675 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:18.168998 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:18.140417 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 138306us
W20260709 13:19:18.143186 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.045578 20755 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:18.175341 20755 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:18.184371 20755 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.187777 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.188140 16664 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:18.112566 20766 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:18.189528 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:18.182680 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 228966us
I20260709 13:19:18.196651 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:18.190841 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 246897us
I20260709 13:19:18.112766 20521 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:18.195653 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.199447 16743 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:18.196887 16737 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:18.197732 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:18.201167 16664 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:18.099553 20738 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.211136 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.229187 16668 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:18.211194 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.230084 16740 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:18.231004 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.211226 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.212136 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.232283 17374 leader_election.cc:400] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:18.232751 16736 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.212970 20550 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.213701 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 547) took 2424 ms. Trace:
I20260709 13:19:18.214185 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.237614 16738 rpcz_store.cc:276] 0709 13:19:15.789760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.792448 (+  2688us) service_pool.cc:224] Handling call
0709 13:19:15.792715 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.351550 (+558835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.213467 (+1861917us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400049a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:18.213635 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238953344}
I20260709 13:19:18.238826 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.239527 16738 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.240176 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.240875 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.214775 16590 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:19:18.215453 16664 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:18.216418 20632 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.216773 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.217556 20593 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:18.217767 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.243367 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2984) took 1047 ms. Trace:
I20260709 13:19:18.225993 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1548) took 2448 ms. Trace:
I20260709 13:19:18.233199 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:18.243902 16590 rpcz_store.cc:276] 0709 13:19:17.196208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.196721 (+   513us) service_pool.cc:224] Handling call
0709 13:19:17.211323 (+ 14602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.214525 (+1003202us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b54002ab0a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb5bdc0 00007f327fbe61c0 00007f327fbe7231 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c
0709 13:19:18.243084 (+ 28559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.243301 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84444928}
I20260709 13:19:18.244270 16666 rpcz_store.cc:276] 0709 13:19:15.777781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:15.827420 (+ 49639us) service_pool.cc:224] Handling call
0709 13:19:15.827691 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.329634 (+1501943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:17.329666 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.329706 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:19:17.329742 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.225724 (+895982us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540029f9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:18.225784 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.225920 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3453716224}
I20260709 13:19:18.248203 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.249220 16741 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:18.235582 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.236608 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.236685 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.237303 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.240983 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.255154 16591 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:18.254415 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.257011 16743 raft_consensus.cc:2368] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:18.254288 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.258039 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.259007 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.259599 16669 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:18.260032 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.260811 16669 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:18.261070 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.196960 20698 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:19:18.265534 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.266448 16589 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:18.266532 16736 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:18.268253 16738 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:18.239444 20631 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:18.243903 20763 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.266012 20017 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:18.272748 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.276367 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.304090 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.274842 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.273206 16670 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:18.257522 20773 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:18.273344 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:18.307624 17412 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:18.264722 20616 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:18.208455 20760 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.336450 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 2979) took 1212 ms. Trace:
I20260709 13:19:18.337067 16592 rpcz_store.cc:276] 0709 13:19:17.123923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.124058 (+   135us) service_pool.cc:224] Handling call
0709 13:19:17.124377 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.702336 (+577959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.336106 (+633770us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54001faed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:18.336367 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327665920}
I20260709 13:19:18.341202 20559 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:18.350131 20774 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:19:18.353048 20764 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:18.353829 20764 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.357170 20773 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.357715 20773 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.359194 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.360003 20773 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.360584 20773 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.361030 20773 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.361316 20595 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:18.362011 20595 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.363206 20764 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.363685 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.364970 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.365679 16743 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.367085 20595 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.367403 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1550) took 1521 ms. Trace:
I20260709 13:19:18.367779 16665 rpcz_store.cc:276] 0709 13:19:16.846287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:16.978028 (+131741us) service_pool.cc:224] Handling call
0709 13:19:16.978326 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:16.996273 (+ 17947us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:17.058749 (+ 62476us) log.cc:844] Serialized 14 byte log entry
0709 13:19:17.235764 (+177015us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:17.235815 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:17.235860 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.367188 (+1131328us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b540001b654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:18.367229 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.367347 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58369,"prepare.queue_time_us":162605,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":51929,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6233,"spinlock_wait_cycles":40723712,"thread_start_us":173798,"threads_started":2,"wal-append.queue_time_us":169388,"wal-append.run_cpu_time_us":1216,"wal-append.run_wall_time_us":1073993}
I20260709 13:19:18.370200 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.370846 16738 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:18.371347 20616 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:18.371693 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.372359 16738 raft_consensus.cc:2368] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:18.372236 20616 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.373144 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.374095 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.245990 20593 consensus_queue.cc:1243] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 7.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:18.375530 16736 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:18.229254 20521 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.376487 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.377542 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.378106 16736 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:18.271884 20635 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:18.311713 20738 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.389120 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.390057 17409 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:19:18.316906 16667 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:18.320541 20758 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.254213 20670 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:19:18.322083 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:18.486857 16737 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:18.487427 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.274096 20631 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.309405 16670 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:18.360002 16741 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:18.381534 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.382088 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.500533 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:18.468254 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:18.439698 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 133079us
I20260709 13:19:18.265239 20735 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:18.262338 20743 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:18.501250 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.512102 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.519608 16738 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.534108 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.521734 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.535326 17416 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:18.540903 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.448923 16483 maintenance_manager.cc:419] P bfe27eaae9f4449e96031a0b1485f80c: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033341
I20260709 13:19:18.552760 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.562943 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.412945 20764 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.411789 20595 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:18.564563 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.396075 20638 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:18.566308 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:18.567063 17415 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:18.570367 16741 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:18.571496 17411 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:19:18.594506 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.595454 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.595687 17415 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:18.307197 20645 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.8, Next index: 8, Last known committed idx: 7, Time since last communication: 0.252s
W20260709 13:19:18.596352 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65351.4us
I20260709 13:19:18.597422 17419 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:19:18.597675 16741 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.597831 17419 pending_rounds.cc:85] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 9
I20260709 13:19:18.598192 17419 pending_rounds.cc:107] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:19:18.598421 17419 raft_consensus.cc:2889] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:18.578387 16737 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:18.600365 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.603099 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.606375 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:18.626533 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:18.627595 20593 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:18.627955 20631 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.628093 20593 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:18.628510 20631 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:18.628880 20631 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.629282 20631 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:18.629050 20593 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.586580 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.579622 20764 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:18.588150 16596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 2988) took 1382 ms (client timeout 1559 ms). Trace:
I20260709 13:19:18.595822 20670 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:18.579249 20595 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.530122 20735 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.607925 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:18.642903 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.611379 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.611574 20645 consensus_queue.cc:1243] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:18.611871 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.619762 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:18.653846 16741 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:18.624019 20760 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.624410 20638 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.578907 20773 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.655165 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.656002 16587 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:18.656540 20638 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.603127 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:18.657398 17413 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:18.647704 16737 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:18.638695 16596 rpcz_store.cc:269] 0709 13:19:17.197492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.197603 (+   111us) service_pool.cc:224] Handling call
0709 13:19:18.579609 (+1382006us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b5400239bd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:18.579713 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85686528}
I20260709 13:19:18.649274 20211 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:18.575417 20521 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:18.659782 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:18.661121 16671 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:18.663532 16668 raft_consensus.cc:1217] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 5.4->[6.5-10.7]   Dedup: 6.5->[8.6-10.7]
I20260709 13:19:18.664065 16668 pending_rounds.cc:85] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 5
I20260709 13:19:18.664289 16668 pending_rounds.cc:107] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:19:18.664547 16668 raft_consensus.cc:2889] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:18.665660 17412 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:18.676082 17412 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:18.688107 16589 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:18.691195 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:18.692847 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.697695 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2806) took 1017 ms. Trace:
I20260709 13:19:18.697880 17415 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.698127 16742 rpcz_store.cc:276] 0709 13:19:17.680199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.680336 (+   137us) service_pool.cc:224] Handling call
0709 13:19:17.695518 (+ 15182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:17.882434 (+186916us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:18.047506 (+165072us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.256500 (+208994us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.258900 (+  2400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:18.258959 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:18.259004 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.594058 (+335054us) spinlock_profiling.cc:243] Waited 4.08 ms on lock 0x7b48002307d8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:19:18.697449 (+103391us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400427054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:18.697509 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.697639 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2114,"prepare.queue_time_us":124561,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":4101,"raft.run_cpu_time_us":14395,"raft.run_wall_time_us":347516,"spinlock_wait_cycles":470535040,"thread_start_us":361122,"threads_started":2,"wal-append.queue_time_us":471165}
I20260709 13:19:18.706337 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.575345 20656 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:19:18.639348 20670 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:18.677949 20645 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:18.658350 20735 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:18.724984 16662 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:18.726739 16662 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:18.709635 20638 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:18.747442 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.719275 20691 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:18.727823 16662 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.755861 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:18.756929 17410 raft_consensus.cc:2368] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:18.704679 20026 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:19:18.721880 20511 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:19:18.753670 20691 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:18.753985 20396 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:18.736869 20645 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:18.779788 20766 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.582s	user 0.005s	sys 0.005s
I20260709 13:19:18.780040 20645 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.780280 20766 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:18.773972 20396 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:18.785499 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:18.797679 17415 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:19:18.800681 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 293681us
W20260709 13:19:18.803335 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.040s
W20260709 13:19:18.811097 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.000s	sys 0.040s
W20260709 13:19:18.811951 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 550897us
I20260709 13:19:18.814210 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.812959 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.818950 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.820299 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.827744 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.831559 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:18.824436 20782 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:18.802211 20691 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.774964 20670 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.837785 20764 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.840399 20755 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:18.827178 20595 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:18.833571 20656 raft_consensus.cc:2764] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:19:18.847939 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.835542 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.838649 20126 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:18.852635 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 181839us
I20260709 13:19:18.838891 20663 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:19:18.865871 16636 leader_election.cc:400] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:18.867704 16636 leader_election.cc:403] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:18.865484 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.870115 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.874118 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:18.876417 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.878657 17409 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:18.851954 20669 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:18.863108 20782 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:18.870831 20126 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.890813 20709 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:18.892925 20631 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:18.878811 20691 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.881134 20233 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:18.915856 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2804) took 1282 ms. Trace:
I20260709 13:19:18.931608 16739 rpcz_store.cc:276] 0709 13:19:17.633198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.633336 (+   138us) service_pool.cc:224] Handling call
0709 13:19:17.633675 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:17.704905 (+ 71230us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:19:17.705224 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.354780 (+649556us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b1800127660. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf4455 00007f32835dc3c7 00007f327fbee9a7 00007f327fbfe0bf 00007f327fbfe038
0709 13:19:18.754301 (+399521us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.756191 (+  1890us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:18.757957 (+  1766us) log.cc:844] Serialized 26 byte log entry
0709 13:19:18.758149 (+   192us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:18.760866 (+  2717us) log.cc:844] Serialized 14 byte log entry
0709 13:19:18.761084 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:18.774406 (+ 13322us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:18.915623 (+141217us) raft_consensus.cc:1685] finished
0709 13:19:18.915643 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:18.915793 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74990,"prepare.queue_time_us":56325,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":82703,"raft.run_cpu_time_us":467,"raft.run_wall_time_us":10398,"spinlock_wait_cycles":15294720,"thread_start_us":638552,"threads_started":2,"wal-append.queue_time_us":318615}
I20260709 13:19:18.916115 20782 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:18.935304 20782 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:18.938513 20396 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.940982 20211 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:18.941439 20211 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:18.943320 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 2172) took 1241 ms. Trace:
I20260709 13:19:18.943747 16663 rpcz_store.cc:276] 0709 13:19:17.701665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.705690 (+  4025us) service_pool.cc:224] Handling call
0709 13:19:18.943114 (+1237424us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b5400346a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:18.943251 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69984512}
I20260709 13:19:18.883344 20737 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:19:18.878054 20664 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:18.927968 20631 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:18.883978 20357 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:18.934559 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:18.953886 16667 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:18.879297 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.872743 20755 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:18.946171 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 253422us
I20260709 13:19:18.959616 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:18.957769 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 78536.8us
I20260709 13:19:18.962936 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.963949 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.965574 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.942416 20211 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:18.972888 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.979543 16667 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
W20260709 13:19:18.981218 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 225882us
I20260709 13:19:18.956396 20737 consensus_queue.cc:1243] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:18.984193 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.987015 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.987871 16589 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:18.989861 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:18.996057 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:18.998031 16666 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 13. Ops: []
I20260709 13:19:18.964876 20664 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:18.999152 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:18.999486 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:18.963197 20766 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:19.004627 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 108302us
I20260709 13:19:19.004855 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.965351 20233 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:19.003736 16666 raft_consensus.cc:2368] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12.
I20260709 13:19:19.009250 16666 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:18.971225 20755 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:19.006014 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:18.963719 20638 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:18.996762 20753 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:19.002017 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:18.970042 20770 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:19.010288 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:19.012569 16669 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:19.011912 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:19.013368 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:19.014349 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:19.002465 20761 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:19:19.016804 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:19.002017 20642 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:19.010658 20211 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:19.033465 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:19.061666 16708 leader_election.cc:400] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12.
I20260709 13:19:19.062214 16708 leader_election.cc:403] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:19:19.022426 16667 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:19.006991 20593 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:19.054833 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:19.068821 20755 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:19.000305 20709 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:19:19.015929 20737 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:19.014019 20664 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:19.016498 20770 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:19.071542 20760 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:19.189160 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:19.056746 20790 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:19.070966 20020 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12.
I20260709 13:19:19.040097 20735 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.036654 20782 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:19.212677 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.116087 20593 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:19.071048 20396 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 13:19:19.075525 20753 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.138s
I20260709 13:19:19.077275 20573 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.145s
W20260709 13:19:19.276952 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 222827us
I20260709 13:19:19.189205 20760 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:19.196058 20770 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:19.260444 20675 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:19.254031 20790 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:19.347225 20806 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:19:19.299119 20760 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:19.322492 20357 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:19.335065 20020 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:19.252873 20638 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:19.388048 16740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.023s	sys 0.031s
W20260709 13:19:19.409512 16740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.023s	sys 0.032s
I20260709 13:19:19.410708 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1370) took 1056 ms. Trace:
I20260709 13:19:19.411504 16740 rpcz_store.cc:276] 0709 13:19:18.354634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.380327 (+ 25693us) service_pool.cc:224] Handling call
0709 13:19:18.380615 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.380826 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:18.381043 (+   217us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.410230 (+1029187us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.410287 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:19.410356 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.410452 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.410628 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":536,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":10071,"thread_start_us":1028795,"threads_started":1,"wal-append.queue_time_us":392532}
W20260709 13:19:19.420953 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 366153us
I20260709 13:19:19.385866 20678 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:19.433043 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:19.389736 20020 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:19.424886 20669 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:19.374109 20664 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.435149 20678 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:19.387141 20593 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:19.466322 20233 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:19.526578 20792 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:19:19.527246 20020 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:19.535079 20808 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:19:19.513610 20737 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:19.462445 20790 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.539024 20782 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:19.589262 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 288964us
I20260709 13:19:19.539250 20233 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.625765 16665 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:19.553025 20760 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:19.541225 20678 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.630187 20233 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:19.651386 20680 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:19.669008 20737 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.674445 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 446) took 1187 ms. Trace:
I20260709 13:19:19.675251 17413 rpcz_store.cc:276] 0709 13:19:18.487018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.487155 (+   137us) service_pool.cc:224] Handling call
0709 13:19:19.673631 (+1186476us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54004b6254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:19.674038 (+   407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389240576}
I20260709 13:19:19.676024 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 453) took 1016 ms. Trace:
I20260709 13:19:19.676609 17415 rpcz_store.cc:276] 0709 13:19:18.659237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.659486 (+   249us) service_pool.cc:224] Handling call
0709 13:19:19.675532 (+1016046us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b54004e5554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:19.675812 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82102016}
I20260709 13:19:19.679962 17414 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:19.680261 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 794148us
I20260709 13:19:19.688512 20678 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:19.694080 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1587) took 1501 ms. Trace:
I20260709 13:19:19.694696 16664 rpcz_store.cc:276] 0709 13:19:18.192483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.356451 (+163968us) service_pool.cc:224] Handling call
0709 13:19:18.356794 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:18.828140 (+471346us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:19.390712 (+562572us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.592839 (+202127us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:19.594152 (+  1313us) log.cc:844] Serialized 26 byte log entry
0709 13:19:19.594392 (+   240us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:19.594473 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.594636 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:19.693518 (+ 98882us) raft_consensus.cc:1685] finished
0709 13:19:19.693551 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.693825 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":424564,"prepare.queue_time_us":504835,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":234069,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":782728,"threads_started":3,"wal-append.queue_time_us":165599}
W20260709 13:19:19.817109 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.824162 17410 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:19.824872 20701 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:19.825842 20701 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:19.828493 16635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.004s	sys 0.020s
W20260709 13:19:19.829388 16635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.004s	sys 0.020s
I20260709 13:19:19.831637 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:19.832562 16561 connection.cc:256] RPC call timeout handler was delayed by -1.3273686600000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2033s for a response
I20260709 13:19:19.832952 17416 raft_consensus.cc:2368] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
W20260709 13:19:19.833549 16561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.4:41455 timed out after 1.714s (SENT) [suppressed 17 similar messages]
W20260709 13:19:19.833597 17416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 450) took 1266 ms (client timeout 1577 ms). Trace:
I20260709 13:19:19.836148 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:19.843755 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 162319us
I20260709 13:19:19.844173 20761 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:19.847069 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.004s	sys 0.022s
W20260709 13:19:19.847597 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.004s	sys 0.022s
I20260709 13:19:19.848675 20709 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:19.834225 17416 rpcz_store.cc:269] 0709 13:19:18.567563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.830981 (+1263418us) service_pool.cc:224] Handling call
0709 13:19:19.833514 (+  2533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:19:19.832321 20701 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:19.861475 20586 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
W20260709 13:19:19.879053 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45537.9us
I20260709 13:19:19.880584 20760 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:19.891547 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44700.6us
I20260709 13:19:19.891884 20761 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:19.893922 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:19.899796 16737 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:19.904206 20760 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:19.905192 20760 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:19.907826 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:19.908933 16738 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:19.911911 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:19.915488 16743 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:19.918283 16741 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:19.922925 20738 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:19.923527 20738 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:19.924491 20825 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:19.925710 20825 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.931437 20738 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:19.932418 16745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1342) took 1872 ms (client timeout 2036 ms). Trace:
W20260709 13:19:19.939277 16745 rpcz_store.cc:269] 0709 13:19:18.059835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.102686 (+ 42851us) service_pool.cc:224] Handling call
0709 13:19:19.931897 (+1829211us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54001c7354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:19.932180 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605075200}
I20260709 13:19:19.938937 20825 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:19.941004 20640 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:19.935906 20830 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:19:19.941620 20640 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:19.941694 20760 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:19.943578 20738 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:19.947014 20720 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:19.948189 20720 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:19.950064 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1598) took 1282 ms. Trace:
I20260709 13:19:19.953265 20760 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:19.954044 20126 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:19.957062 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 523935us
W20260709 13:19:19.936337 20793 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.639s	user 0.008s	sys 0.000s
W20260709 13:19:19.936584 20211 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.016s
W20260709 13:19:19.960253 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 368711us
I20260709 13:19:19.958618 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:19.958510 20524 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:19.976373 16738 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:19.975986 20765 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:19.979303 20765 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:19.979746 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:19.974195 16634 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:19.965086 20720 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:19.954089 20760 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:19.961508 20669 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:19.980794 20789 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:19.950520 16671 rpcz_store.cc:276] 0709 13:19:18.667138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.667305 (+   167us) service_pool.cc:224] Handling call
0709 13:19:18.667729 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.668009 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:19:19.372961 (+704952us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.426340 (+ 53379us) log.cc:844] Serialized 14 byte log entry
0709 13:19:19.949040 (+522700us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b18004829e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:19:19.949353 (+   313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.949436 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:19.949540 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.949671 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:19.949885 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":899710,"prepare.queue_time_us":653330,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":530916,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":5188,"spinlock_wait_cycles":63572608,"thread_start_us":376610,"threads_started":2,"wal-append.queue_time_us":163865}
I20260709 13:19:19.988479 20669 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:19.991618 20821 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:19.988279 20640 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:19.998028 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:19.991647 20765 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:19.998811 20819 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:19.999646 20820 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:20.000353 20805 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:20.024168 20760 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.018692 20735 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:20.028642 20819 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:20.047338 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88943.5us
I20260709 13:19:20.049741 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:20.050515 17410 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:20.051555 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:20.053332 20819 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:20.053992 20831 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:20.055459 20554 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:20.055850 20554 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:20.057168 20554 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.058636 20554 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:20.064991 20811 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.065659 20811 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:20.068203 17409 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 797) took 1510 ms (client timeout 1649 ms). Trace:
W20260709 13:19:20.068645 17409 rpcz_store.cc:269] 0709 13:19:18.557309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:18.561427 (+  4118us) service_pool.cc:224] Handling call
0709 13:19:20.067873 (+1506446us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400681354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:20.068035 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284745472}
I20260709 13:19:20.070959 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.080291 16596 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:20.081676 20720 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:20.060838 20663 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:20.082996 20663 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.083401 20663 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:20.087754 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.072582 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:20.082410 20720 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.094255 20789 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.094630 20720 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.094776 20789 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:20.075103 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:20.099701 17419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 657) took 1091 ms. Trace:
I20260709 13:19:20.100189 17419 rpcz_store.cc:276] 0709 13:19:19.008594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.089586 (+ 80992us) service_pool.cc:224] Handling call
0709 13:19:19.677841 (+588255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.678403 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:19.678433 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:19.678498 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:19.678551 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:19.678650 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:20.099324 (+420674us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54004b6250. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:20.099486 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120818432}
I20260709 13:19:20.101292 20763 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:20.103643 20396 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:20.094981 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1779686600000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21989999999999998s for a response
I20260709 13:19:20.115397 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.116304 16593 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:19:20.124886 20616 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.125819 20616 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.126186 20815 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.127180 20755 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.128044 20755 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.130198 16738 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:19:20.130561 20755 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.082724 16662 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:20.104349 20396 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.139676 20738 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.158039 20770 consensus_meta.cc:287] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.779s	user 0.021s	sys 0.004s
I20260709 13:19:20.159534 20664 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.160353 20680 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.158079 20638 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.169523 20680 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:20.169917 20638 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.159580 20675 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.170595 20616 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.171123 20675 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:20.142086 20735 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:20.176426 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:20.195361 16738 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
W20260709 13:19:20.178117 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 81277.1us
W20260709 13:19:20.199451 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.194043 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:20.194229 20770 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.213203 16594 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:20.173784 20524 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.214388 20770 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.214564 20524 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.132839 20815 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.172578 20638 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.173559 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.175642 20669 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.175343 20735 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.227036 20738 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.227842 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:20.230149 20675 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.226176 20680 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.242650 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:20.209672 20640 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:20.222126 20789 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:20.245852 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 115215us
I20260709 13:19:20.246476 16745 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:20.252315 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:20.261085 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.262287 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.263866 16594 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:20.264385 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.264995 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.265250 16587 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:20.265631 16594 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:20.266289 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.267468 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.269207 16587 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 13. Ops: [11.7-11.7]
I20260709 13:19:20.272158 20656 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:20.272719 20656 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.272779 20735 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.273834 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.275496 20738 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.276249 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:20.276580 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:20.277163 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.278594 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.280836 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.281541 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.273329 20656 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.270717 16587 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:20.280177 16592 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:19:20.286540 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.288327 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:20.316978 16594 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
W20260709 13:19:20.320999 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44520.6us
I20260709 13:19:20.332402 16738 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:20.332562 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.333690 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.335080 16596 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:20.286381 20524 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.338099 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.295967 20669 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.332953 16595 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:20.343250 17416 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: [7.7-8.8]
I20260709 13:19:20.245050 20664 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.340775 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.345866 16745 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:20.345041 16662 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:20.347043 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.351046 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.347153 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:20.347549 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 152281us
I20260709 13:19:20.347997 20632 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:20.353699 17419 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:20.354462 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1579) took 2934 ms. Trace:
I20260709 13:19:20.355463 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:20.358006 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:20.354811 16662 rpcz_store.cc:276] 0709 13:19:17.412263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:17.473168 (+ 60905us) service_pool.cc:224] Handling call
0709 13:19:17.473575 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:18.724862 (+1251287us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54001af520. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb5bdc0 00007f327fbe61c0 00007f327fbe7231 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c
0709 13:19:20.081837 (+1356975us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b58006575d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:19:20.346327 (+264490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.346508 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2945696896}
I20260709 13:19:20.364317 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:20.365841 16591 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:20.367019 20521 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.367558 20521 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:20.372361 20650 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:19:20.372562 16636 connection.cc:256] RPC call timeout handler was delayed by -1.5314574829999401s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 13:19:20.373044 16636 connection.cc:256] RPC call timeout handler was delayed by -1.5296493840000949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 13:19:20.373471 16636 connection.cc:256] RPC call timeout handler was delayed by -1.4855493840000236s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
I20260709 13:19:20.376672 20845 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:20.341598 20805 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.377878 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:20.391866 20631 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.392781 20631 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.394945 20126 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:20.395584 20616 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.396376 20616 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.397260 16561 leader_election.cc:400] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:20.397575 16561 leader_election.cc:403] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:20.397676 20691 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.399439 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:20.392001 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:20.398273 20691 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:20.428076 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.426455 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.429364 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.377357 20845 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:20.398696 20616 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.433928 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:20.434209 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:20.427115 20680 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.395648 20126 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.437144 20753 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:20.439268 20669 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:20.440090 20805 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.449116 20664 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.453207 20691 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:20.459232 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.455003 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 655) took 1447 ms. Trace:
I20260709 13:19:20.456607 20811 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.457347 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.458267 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.457386 17413 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:20.459082 20843 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:20.492897 16670 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:20.490594 17414 rpcz_store.cc:276] 0709 13:19:19.007826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:19.008154 (+   328us) service_pool.cc:224] Handling call
0709 13:19:19.679273 (+671119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:19.680555 (+  1282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:20.454699 (+774144us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54004e5550. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:20.454924 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522965120}
I20260709 13:19:20.404173 20844 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:20.506170 16662 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:20.489528 20675 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.518585 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.491606 20631 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:20.466768 20663 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.499657 20843 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:20.472612 20845 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.495445 20680 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:20.543306 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:20.544061 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 110300us
I20260709 13:19:20.561715 16664 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:20.537732 20843 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:20.568130 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.526899 20586 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:20.529909 20664 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:19:20.591091 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:20.593272 20805 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.575865 20753 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:20.610736 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:20.622509 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.602342 20815 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:20.587235 20586 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:20.651099 20554 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.551s	user 0.004s	sys 0.008s
I20260709 13:19:20.657752 20586 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.659478 16669 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
W20260709 13:19:20.662510 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 112634us
W20260709 13:19:20.663331 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 264083us
I20260709 13:19:20.622452 20805 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:20.666142 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.666596 20670 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:20.667205 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:20.667802 20856 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:20.679165 20663 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.683365 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:20.685874 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.686563 16663 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:20.687155 20396 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.647833 20753 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.693744 20789 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.690738 16668 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:19:20.703526 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 114062us
I20260709 13:19:20.716324 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:20.717304 20521 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:20.722954 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.730248 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.730618 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.731026 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:20.731185 16588 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:20.732012 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.732367 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.732735 16592 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:20.732954 16588 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:20.733091 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.733578 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.733865 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.734750 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.734830 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.735471 16595 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:20.735172 20211 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:20.734939 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.736228 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.736757 16589 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:20.738492 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.738761 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.739741 20789 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:20.740252 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.740159 20211 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:20.743112 20753 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.740294 20789 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.744591 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.745025 20789 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.745343 20845 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:20.746235 16594 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:20.747767 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.748984 16591 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:20.750383 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:20.751624 17412 raft_consensus.cc:2410] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:19:20.744663 20126 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.754964 16595 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:20.758652 16595 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:20.658178 20656 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.769855 16589 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:20.770370 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.777014 16745 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:20.771518 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.780308 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.784466 16671 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:19:20.785579 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.786339 16671 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:20.765043 20675 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:20.790175 16666 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
W20260709 13:19:20.791822 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 75653us
I20260709 13:19:20.793202 16663 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:20.794358 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.801788 16596 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:20.799805 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:20.808073 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.810302 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.817116 16589 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.820091 16671 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:20.820240 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.799252 20675 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:20.822120 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.825670 16595 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:20.824725 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.831092 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.831976 16595 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:20.834831 20638 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:20.835064 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:20.827346 20675 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:20.838167 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:20.843621 20766 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.844182 20766 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.846288 20766 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.849093 16663 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:20.851683 20632 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:20.853137 20650 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:19:20.853539 20650 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.853863 20650 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:20.840986 20814 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:20.855518 20814 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:20.855868 20814 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:20.856019 20761 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:20.856441 20761 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:20.856825 20761 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:20.851244 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:20.860913 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:20.837306 20638 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:20.837110 20547 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:20.868616 20864 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:19:20.870078 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:20.874212 16668 raft_consensus.cc:2410] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:19:20.875178 20547 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.877725 20547 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.864428 20632 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:20.887459 20632 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.887887 20632 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:20.888788 20026 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:20.894047 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59087.1us
I20260709 13:19:20.830354 20811 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:19:20.907420 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:20.913638 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:20.916941 16592 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:20.910195 20815 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:20.938053 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:20.939018 16591 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
W20260709 13:19:20.948271 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:20.919377 20547 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:20.900645 20843 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:20.967873 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:20.944247 20815 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:20.971197 20815 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:20.971871 20815 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:19:20.974463 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 79667.8us
W20260709 13:19:20.975353 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 145839us
I20260709 13:19:20.957347 20547 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:20.952536 20656 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:20.984869 16595 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:20.978407 20811 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:20.996284 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 83207us
I20260709 13:19:20.999325 20357 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:20.999917 20357 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.000288 20593 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:19:21.001188 20593 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:21.001529 20593 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:21.013327 20843 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:20.988924 20547 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:21.019556 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.024572 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:21.027052 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:21.018381 20811 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.040940 20811 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.043989 20650 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.045677 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.046552 20845 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:21.046267 20650 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.000291 20357 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:20.941404 20858 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:19:21.048537 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:21.092182 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65198.1us
I20260709 13:19:20.999388 20860 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:19:21.002418 20593 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.054683 20616 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:21.114307 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.117691 20845 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:19:21.117770 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69323.8us
I20260709 13:19:21.057982 20632 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.026667 20669 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:21.126717 20858 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:21.127310 20845 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:21.153964 20691 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:19:21.168973 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.171087 20763 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:19:21.173250 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.180855 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 161409us
I20260709 13:19:21.102540 20675 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.126106 20026 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:19:21.135397 20737 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:21.196259 16589 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
W20260709 13:19:21.208835 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 115901us
I20260709 13:19:21.209159 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:21.198704 20843 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.199044 20805 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.203500 20766 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.224771 16592 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
W20260709 13:19:21.229403 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.235121 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.214890 20616 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:21.239720 16670 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 6. Current term is 7. Ops: []
I20260709 13:19:21.247848 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.216542 20761 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.254096 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:21.269546 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.272245 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:21.259889 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.241957 20669 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:21.210355 20737 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.214478 20814 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.280254 20632 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.280876 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:21.282591 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:21.289649 16742 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:21.252084 20760 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.294375 16669 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:21.230191 20805 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.295809 20670 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:21.302515 20632 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.302686 20761 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.242741 20766 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.303297 20632 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.305845 20638 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.306559 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:21.306480 20638 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.307327 20547 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.308590 20642 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:21.308717 20638 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:21.309046 20642 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:21.309293 20650 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:21.309640 20547 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:21.310514 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 105569us
I20260709 13:19:21.312633 20663 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:21.310137 20642 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:21.313488 20870 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:19:21.313822 20642 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:21.316830 20670 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:21.317970 20670 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.320914 20701 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:21.321434 20701 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:21.321730 20872 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:19:21.323370 20663 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:21.324501 20814 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:21.324429 20663 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.325389 20814 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.325945 20814 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.328222 20675 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.288057 20211 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:21.335590 20632 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.336336 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:21.336968 16589 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:21.284685 20586 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:19:21.340008 20586 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:21.340505 20586 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:21.340112 20760 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.341439 20586 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.342833 20753 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.343580 20753 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.344254 20790 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:21.344708 20790 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:21.345606 20790 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.325502 20593 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:21.322504 20701 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.348585 20843 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:21.342658 20586 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:21.250103 20879 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:19:21.372617 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.373245 16663 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:21.360304 20616 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:21.349896 20632 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:21.375672 20632 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:21.376960 20877 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260709 13:19:21.377493 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 139757us
I20260709 13:19:21.378372 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.379974 20843 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:21.380590 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.380776 20547 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:21.382473 20879 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:21.382836 20879 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:21.383083 20760 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:21.384122 20805 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:21.383773 20879 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.385221 20879 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:21.385071 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.385849 20753 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:21.387152 20814 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:21.355263 20701 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:21.389456 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.150841 20874 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Committing config change with OpId 9.8: config changed from index 2 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:21.390255 16664 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:21.246106 20845 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.391669 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.391782 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.394117 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.394645 16667 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:21.399149 20790 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:21.405786 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:21.406627 16591 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:21.405786 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.408432 16668 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:21.415652 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 76122.9us
I20260709 13:19:21.411521 20357 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.424577 16589 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
W20260709 13:19:21.427089 20871 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.273s	user 0.003s	sys 0.001s
I20260709 13:19:21.425258 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:21.433815 20616 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.434760 20396 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:21.435472 20396 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.435809 20396 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:21.430706 16667 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:21.439188 20766 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:21.444415 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.442451 20843 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:21.448829 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.450417 16636 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:21.450884 20357 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:21.451565 20357 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.451818 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.452523 20357 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:21.456411 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75915.5us
I20260709 13:19:21.456539 20632 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.457438 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:21.458230 20632 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:21.459044 20843 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:21.459599 20843 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:21.460033 20843 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:21.470973 20869 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.459s	user 0.003s	sys 0.000s
I20260709 13:19:21.472725 16561 leader_election.cc:400] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:21.489228 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:21.493489 20882 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:19:21.497622 16668 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:21.473893 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.498940 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:21.499883 16594 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:21.500377 16663 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
W20260709 13:19:21.504312 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76612.1us
I20260709 13:19:21.505992 20666 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:21.508571 20521 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:21.524463 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:21.492162 20396 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.520702 20631 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:21.513293 16594 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:21.513283 20642 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:21.520730 20521 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:21.544847 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:21.551115 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:21.559684 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85886us
I20260709 13:19:21.537268 20860 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:21.562588 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.562945 20815 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.563854 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:21.566018 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 114299us
I20260709 13:19:21.567816 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:21.572618 16594 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:21.546802 20638 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:19:21.594969 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43942.2us
I20260709 13:19:21.595830 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:21.602087 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.614282 16594 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:21.572935 20663 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:21.593142 20396 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:21.566368 20642 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.621923 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:21.582209 20815 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.592801 20860 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:21.635169 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.639056 16662 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:21.627200 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.616478 20737 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:21.644991 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.604187 20885 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:21.625430 20883 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:21.647601 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.650722 20815 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:21.650588 20642 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:21.562660 20521 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.652575 16745 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:21.635411 20701 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:21.652853 16740 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:21.642127 20670 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:21.636368 20663 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:21.658133 20211 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.665917 20670 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:21.667156 20670 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:21.675567 20890 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 13:19:21.683182 20860 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.688709 20521 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:21.688392 20663 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.689764 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:21.690411 17410 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:21.693007 20883 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:21.693251 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:21.694283 20889 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:21.694092 20883 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.694696 20889 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.695091 20889 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:21.637192 20631 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:19:21.701390 20891 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:19:21.702594 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44207us
I20260709 13:19:21.703440 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.704689 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.725708 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.713629 20691 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:21.731536 20766 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:21.723418 20885 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:21.733140 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.714437 20126 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:21.726763 20867 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:19:21.745488 20126 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.747732 20126 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:21.709725 20735 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:19:21.763626 16589 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:21.765210 20885 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.772691 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:21.773424 20126 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:21.779134 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:21.780264 16489 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 11, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:21.787782 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.789305 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.789983 16669 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:21.790426 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.790938 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.791669 16669 raft_consensus.cc:2368] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:21.791640 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.792083 20843 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.793455 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.794032 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.794600 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.729277 20701 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:21.799888 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 95274us
W20260709 13:19:21.800817 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.801288 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.805716 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.799914 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.806630 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:21.744560 20766 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:21.794106 16666 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:21.809654 20766 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.810480 16671 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:21.814787 20845 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:21.808521 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:21.827597 20766 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:21.828578 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.833348 20843 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:21.834450 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.850731 16667 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:21.851497 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.782612 16489 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 8, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 8 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:21.842549 20889 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:21.868279 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 59834.9us
I20260709 13:19:21.861917 20858 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:21.869437 20753 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:21.881728 20753 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:21.884028 20753 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:21.884873 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.885439 16489 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 8, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:21.892135 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85392.2us
I20260709 13:19:21.898875 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:21.899029 20843 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:21.899863 20901 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:21.900256 20901 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.900578 20901 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:21.906092 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73600.3us
I20260709 13:19:21.907895 16594 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:19:21.908784 16490 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 213) took 33916 ms (client timeout 15000 ms). Trace:
I20260709 13:19:21.909333 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:21.910931 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:21.912025 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.912496 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:21.913803 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.916323 20026 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:19:21.917889 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:21.923177 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:21.928691 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.930070 20874 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:21.930472 20874 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:21.924958 20521 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
W20260709 13:19:21.909307 16490 rpcz_store.cc:269] 0709 13:18:47.910225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:18:47.910753 (+   528us) service_pool.cc:224] Handling call
0709 13:19:21.797320 (+33886567us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:14.165618 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:14.211454 (+ 45836us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:14.211504 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:19:14.211521 (+    17us) tablet.cc:662] Decoding operations
0709 13:19:14.212607 (+  1086us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:14.212715 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:14.212756 (+    41us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:19:14.213755 (+   999us) tablet.cc:701] Row locks acquired
0709 13:19:14.213812 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:19:14.214046 (+   234us) write_op.cc:270] Start()
0709 13:19:14.214224 (+   178us) write_op.cc:276] Timestamp: P: 1783603154213966 usec, L: 0
0709 13:19:14.214248 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:14.323826 (+109578us) log.cc:844] Serialized 3205 byte log entry
0709 13:19:14.565310 (+241484us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327cada0ac 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b
0709 13:19:16.316820 (+1751510us) op_driver.cc:464] REPLICATION: finished
0709 13:19:16.365491 (+ 48671us) write_op.cc:301] APPLY: starting
0709 13:19:16.370506 (+  5015us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:16.400766 (+ 30260us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:16.403911 (+  3145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:16.851992 (+448081us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:16.852223 (+   231us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:19:16.852273 (+    50us) write_op.cc:312] APPLY: finished
0709 13:19:16.852557 (+   284us) log.cc:844] Serialized 65 byte log entry
0709 13:19:16.852726 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.852853 (+   127us) write_op.cc:454] Released schema lock
0709 13:19:16.852899 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:16.853303 (+   404us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:16.853334 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1084709,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":41866,"apply.run_cpu_time_us":3907,"apply.run_wall_time_us":487917,"num_ops":8,"prepare.queue_time_us":150295,"prepare.run_cpu_time_us":72083,"prepare.run_wall_time_us":767343,"raft.queue_time_us":448380,"raft.run_cpu_time_us":13886,"raft.run_wall_time_us":77938,"replication_time_us":2086141,"spinlock_wait_cycles":69890304,"thread_start_us":803809,"threads_started":4,"wal-append.queue_time_us":787677,"wal-append.run_cpu_time_us":57419,"wal-append.run_wall_time_us":2720691}]]}
I20260709 13:19:21.948334 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.963451 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.964350 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.964881 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:21.964985 16738 raft_consensus.cc:2376] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11. Re-sending same reply.
I20260709 13:19:21.926668 20906 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:21.972473 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.966831 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:21.975049 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.975667 16738 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:21.971191 20906 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:21.982373 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.984573 16742 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:21.931371 20874 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:21.989135 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:21.991923 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:21.997462 17415 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
W20260709 13:19:21.989276 20910 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:19:22.002926 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:21.987959 20906 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:22.000458 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.013516 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.014057 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:22.014158 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.015035 17415 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:22.015456 16664 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:22.020557 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.016249 20701 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:22.022909 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.024940 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.020105 20874 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:22.027530 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.029707 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.030422 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:22.031436 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:22.032019 16663 raft_consensus.cc:2376] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10. Re-sending same reply.
I20260709 13:19:22.036386 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.037112 16663 raft_consensus.cc:2368] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:22.039611 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:22.042068 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.042647 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:22.043176 16738 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:22.043454 17415 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:22.046911 20825 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:19:22.050266 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:22.051206 17414 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:22.057385 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:22.073071 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.073702 16738 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:22.072255 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:22.077843 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:22.074923 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.089010 20593 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:21.990478 16489 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 9, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 9 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:22.094359 20593 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:22.072731 20825 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:19:22.095549 20825 consensus_queue.cc:1243] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:19:22.080598 16742 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:22.100294 16595 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:22.099063 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.101392 16592 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:22.098591 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:22.103745 16664 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:22.103113 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.107264 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:22.073127 17414 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:22.115629 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:22.117277 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.117690 16667 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:22.105734 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.119611 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.120797 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.122732 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.122812 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.089277 20906 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:22.095604 16489 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 7, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 7 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:22.125329 20906 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.125417 16489 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:22.126997 20915 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.158s	user 0.004s	sys 0.000s
W20260709 13:19:22.082252 20912 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:19:22.051328 20901 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:22.131580 20901 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.132257 20874 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.081693 20889 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:22.132757 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.134130 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.135010 16738 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:22.139003 20586 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:22.140350 17415 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:22.140345 16591 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:22.150012 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.150081 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:22.139513 20586 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.153990 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.151508 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.154994 16740 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:22.139057 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.153112 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.156248 17420 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:22.156106 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.156006 16592 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:19:22.153672 20586 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.157424 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.160061 20766 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:22.160558 20766 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:22.160970 20766 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.164036 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.164779 16736 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:22.165737 16741 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:19:22.167258 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.169121 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.173893 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.175354 16742 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:22.177153 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.179207 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.181572 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.182461 16743 raft_consensus.cc:2410] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:22.179198 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:22.183269 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63719.7us
I20260709 13:19:22.175523 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.197867 16592 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:22.198484 16738 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:22.200021 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.224315 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.225129 16742 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:22.147423 20858 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:22.153750 20701 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:19:22.216349 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.229084 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.229616 16738 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:22.216989 20593 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.217087 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:22.217514 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70424.7us
I20260709 13:19:22.219359 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.221028 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.232475 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.232662 16743 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:22.201464 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:22.224895 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.233618 16589 raft_consensus.cc:2368] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:22.233985 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.234313 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.202253 16489 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 6 to 11, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 11 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:22.225900 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.226608 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:22.230274 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.230641 17414 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:19:22.240084 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.240818 16588 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:22.240718 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.241689 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.241914 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.243000 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.243096 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.139640 20825 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:19:22.244033 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.234560 16596 raft_consensus.cc:2376] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14. Re-sending same reply.
I20260709 13:19:22.244285 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.245141 16745 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:19:22.245185 16596 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:22.246119 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.242782 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.247879 16588 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:22.251216 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.251539 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.251955 16589 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:22.231998 20521 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 13:19:22.253072 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53121.8us
I20260709 13:19:22.243227 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.253973 16595 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:22.228346 20858 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:22.253170 20521 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:22.254664 20858 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:22.254813 20521 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:22.232602 20906 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.237653 16744 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:19:22.254689 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.256950 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.257603 16589 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:22.231047 16740 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:22.261947 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.262092 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.262387 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.262720 16738 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:22.263219 16739 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:22.263475 16594 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:22.264019 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.264523 16736 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:22.255882 20521 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:22.274537 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48010.1us
I20260709 13:19:22.275363 20521 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.266024 16741 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:22.275812 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.277024 20670 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:22.277386 20670 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:22.277737 20670 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:22.279372 20790 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:22.279829 20790 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:22.280205 20790 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:22.266556 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.283771 16588 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:22.269538 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:22.269872 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.289885 16593 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:22.293251 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.295320 16744 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:22.296712 16742 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: [11.6-14.7]
I20260709 13:19:22.300024 16740 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: [9.6-10.7]
I20260709 13:19:22.294946 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:22.301095 16594 raft_consensus.cc:2376] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10. Re-sending same reply.
I20260709 13:19:22.303098 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:22.304657 16742 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:19:22.305172 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.309374 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:22.310711 16738 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:22.310671 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.319447 16667 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:22.323177 20026 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:22.323551 16736 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:22.327962 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:22.331208 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:22.331847 16737 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:22.295822 20906 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:22.345218 20906 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:22.345708 20906 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:22.347468 20631 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:22.347633 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:22.347932 20631 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:22.348361 20631 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:22.348994 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:22.349452 20763 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.350167 20763 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.350286 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.261911 20766 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:22.352082 17420 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:22.354357 20593 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.332259 20755 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:19:22.333182 16742 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:22.330130 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.334414 20396 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:22.308422 20638 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:22.336223 16489 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 10, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:22.338932 20858 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:22.343657 20642 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.377887 20755 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.298002 20663 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:19:22.346587 20906 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:22.396174 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66087.2us
I20260709 13:19:22.397454 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:22.396481 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46253.2us
I20260709 13:19:22.417359 16591 raft_consensus.cc:2368] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
W20260709 13:19:22.565984 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 167579us
W20260709 13:19:22.576772 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 283560us
W20260709 13:19:22.588130 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 290990us
I20260709 13:19:22.541858 20766 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.697594 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:22.698493 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:22.702217 20845 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.703773 20845 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:22.704471 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 307089us
I20260709 13:19:22.702021 16489 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:22.705734 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.705758 20845 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.707252 20761 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.716902 16743 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:22.707176 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.707479 20884 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:19:22.711110 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 416630us
I20260709 13:19:22.707814 16489 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 5, term changed from 5 to 10, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to <none>, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 10 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:22.723670 20631 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:22.728423 20026 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.735147 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:22.744136 20631 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:22.745945 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.748915 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.755211 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.767719 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:22.727234 20586 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.570s	user 0.015s	sys 0.000s
I20260709 13:19:22.793912 20026 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.840751 20026 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:22.838749 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 71141.2us
I20260709 13:19:22.862676 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:22.873857 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 176363us
I20260709 13:19:22.741909 20663 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:22.765545 20761 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:22.766625 20888 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:22.887792 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.703801 20670 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:22.883106 20026 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.778301 20919 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:22.893983 20753 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:19:22.871683 20763 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:19:22.905218 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62861.3us
I20260709 13:19:22.906944 20026 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:22.908198 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 173137us
I20260709 13:19:22.913147 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.913192 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:22.880267 20884 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:22.922304 20026 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:22.900101 20845 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.922034 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:22.864828 20586 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:22.930105 16668 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:22.932818 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:22.929306 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 183671us
I20260709 13:19:22.946161 16708 leader_election.cc:400] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10.
I20260709 13:19:22.947180 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.889845 20663 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:22.914856 20638 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:22.910751 20763 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:22.952692 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:22.958416 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45312.8us
I20260709 13:19:22.959502 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.943305 20664 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:22.966022 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.951130 20761 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:22.949294 20701 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:22.957212 20763 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.963266 20357 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:19:22.986083 20858 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.987564 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.988396 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:22.988766 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.943903 20845 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.005242 20887 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:23.006146 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:23.007014 20926 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 13:19:23.008081 20921 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 13:19:23.011053 20396 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:23.011648 20396 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.013787 20396 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.019474 20924 raft_consensus.cc:2764] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:19:23.022078 20586 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:23.022533 20925 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:23.023074 20663 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.024441 20026 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.024453 20663 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:23.031299 20928 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:23.032603 20922 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.117s	user 0.000s	sys 0.004s
I20260709 13:19:22.961512 20931 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:23.035852 20931 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:23.036285 20931 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:22.968714 16489 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.040199 16589 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:23.043126 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54437.3us
I20260709 13:19:23.043918 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.045079 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:22.968891 20766 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.049489 20888 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:22.959127 20638 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.063990 20638 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:22.977082 20701 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:23.064641 20701 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:22.983001 20889 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.005295 20934 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.054358 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:23.055086 20825 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:23.056140 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.069329 20825 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.069687 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:23.070602 20825 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:23.070986 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:23.071362 20825 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:22.975040 20631 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:23.072010 20889 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:23.074653 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 115213us
I20260709 13:19:23.076375 20889 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:19:23.079635 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.060585 20928 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:23.084492 20631 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.084223 20126 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:23.083312 20735 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:23.061944 20888 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:23.085337 20735 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:23.086287 20888 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:23.081072 20790 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.800s	user 0.017s	sys 0.000s
I20260709 13:19:23.086468 20735 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.087530 20790 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.088205 20735 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:23.089399 20670 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.090736 16664 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:23.075503 20845 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.104071 20126 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:23.105046 20126 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.106272 20126 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.108453 20126 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:23.040319 16489 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 9, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), pending config changed. New cstate: current_term: 9 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } } pending_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.109911 16489 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 7, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 7 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.111804 16489 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 6 to 11, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 11 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:23.113189 20631 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:23.114594 20631 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:23.115584 20026 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:23.116034 20026 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:23.117087 20026 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.119593 20858 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:23.120746 20593 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:23.121130 20593 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:23.121979 20593 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.072605 20825 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.123286 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.123924 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:23.125388 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:23.127481 20931 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.128876 20026 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.143059 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:23.147358 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:23.157678 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.147482 20858 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.147814 20593 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.144686 20790 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.155406 20845 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:23.171437 20885 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.113620 16489 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 2 to 8, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.202627 16664 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:23.171967 20889 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.210613 16489 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 8, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.212430 16489 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 5, term changed from 4 to 7, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4), ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 7 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:23.224123 16664 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:23.201841 20858 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:19:23.208498 20845 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:19:23.236799 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 331839us
I20260709 13:19:23.237229 16664 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:23.244992 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.262960 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.266566 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:23.220170 20889 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.275578 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:23.275414 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 269403us
I20260709 13:19:23.278297 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:23.279083 16663 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.267163 20790 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:23.287276 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 449909us
I20260709 13:19:23.288004 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.236447 20763 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.291531 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:23.295032 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.299775 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.293570 20931 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.301229 20936 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:23.301950 20790 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.303001 20790 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.307050 20888 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:23.323076 16489 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 5, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 5 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.325062 16489 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 4 to 11, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 11 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:23.327234 16591 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:23.328333 20763 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.329043 20763 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:23.329419 20763 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:23.336292 20889 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:23.291738 20766 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:23.338109 20670 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.281188 20930 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:23.339617 20930 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.339233 20521 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:23.340514 20930 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.341048 20521 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:23.341609 20521 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.344646 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:23.344787 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:23.345556 16667 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
W20260709 13:19:23.349530 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49844.2us
I20260709 13:19:23.350790 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.352365 20663 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.359844 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:23.360788 16667 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:23.362874 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 213401us
I20260709 13:19:23.364037 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:23.365139 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 215342us
I20260709 13:19:23.366104 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.370045 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:23.371292 16663 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:23.330235 20763 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.373623 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:23.378361 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.382558 20920 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:23.383255 20920 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.384184 20920 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.389037 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.389822 16740 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:19:23.390208 16740 pending_rounds.cc:85] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 6
I20260709 13:19:23.390434 16740 pending_rounds.cc:107] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:23.390695 16740 raft_consensus.cc:2889] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.392163 16741 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:23.393548 16743 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:19:23.393919 16743 pending_rounds.cc:85] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 4
I20260709 13:19:23.394105 16743 pending_rounds.cc:107] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:19:23.394302 16743 raft_consensus.cc:2889] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.395939 16742 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:19:23.423436 16742 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 13. Ops: []
I20260709 13:19:23.424609 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:23.426874 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:23.433897 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:23.441736 16591 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:23.443816 20919 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:23.448221 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.452262 20919 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:23.452796 20919 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:23.453563 20919 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:23.449025 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:23.456866 20929 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.457446 20929 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.459856 20929 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.461421 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:23.462087 16589 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
W20260709 13:19:23.463589 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.464298 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:23.465572 20929 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.466005 20929 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.466420 20929 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.451792 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.342633 20521 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:23.474795 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 186853us
I20260709 13:19:23.477000 16667 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:23.353910 20670 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:23.499554 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:23.501199 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.372318 20396 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:23.372143 20663 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:23.374709 20937 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.529079 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:23.529224 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 156745us
I20260709 13:19:23.354127 20701 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.502146 20126 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:23.536790 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:23.539162 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:23.538054 17374 leader_election.cc:400] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
W20260709 13:19:23.530092 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 165395us
I20260709 13:19:23.561548 17374 leader_election.cc:403] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:23.551460 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:23.564941 16594 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
W20260709 13:19:23.569813 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.520119 20670 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.549443 20924 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:23.547384 20638 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:23.548470 20126 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.566251 20790 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.543990 20396 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.569869 20906 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.547540 20944 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:23.535475 20945 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:23.542062 20663 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.575364 20766 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:23.572345 20885 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:19:23.565327 20928 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:19:23.590827 20638 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:23.623776 20670 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:23.609887 20906 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.604846 20924 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:23.625753 20663 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:23.606470 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44497.5us
I20260709 13:19:23.636902 20936 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:23.598727 20126 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.650743 16664 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
W20260709 13:19:23.662257 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 126347us
W20260709 13:19:23.665673 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 91233.5us
I20260709 13:19:23.682612 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:23.663147 20735 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.001s
I20260709 13:19:23.659408 20638 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:23.697639 16736 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:23.698237 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.695721 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:23.698992 16739 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:23.609506 20790 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.700865 16592 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:23.706218 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:23.707010 16736 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:23.707325 17410 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:19:23.705137 16589 pending_rounds.cc:85] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 4
I20260709 13:19:23.707455 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:23.702838 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:23.707854 16589 pending_rounds.cc:107] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:19:23.701493 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:23.707993 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:23.708056 16592 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:23.709350 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.710183 16739 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:23.708220 16589 raft_consensus.cc:2889] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:23.704990 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:23.711277 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.707183 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.712271 16742 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:23.618628 20944 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.644044 20631 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:19:23.620644 20945 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:23.691746 20753 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:19:23.737718 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 147998us
W20260709 13:19:23.740423 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 202548us
I20260709 13:19:23.740433 16592 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:23.740968 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.741480 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.741591 16741 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:23.748533 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:23.753931 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:23.754518 17414 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:23.758936 20650 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:23.759492 20650 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.761057 17414 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:23.761498 20650 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:23.768654 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:23.773954 16736 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:23.776235 17420 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:23.780232 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:23.793154 20951 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:19:23.799356 17374 connection.cc:256] RPC call timeout handler was delayed by -1.4165159839999433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1688s for a response
W20260709 13:19:23.799911 17374 connection.cc:256] RPC call timeout handler was delayed by -1.3474015879999115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1778s for a response
W20260709 13:19:23.800294 17374 connection.cc:256] RPC call timeout handler was delayed by -1.0254605229999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 13:19:23.801045 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:23.795300 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:23.753343 20753 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:23.805959 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49450.9us
I20260709 13:19:23.775041 20521 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:23.771306 20957 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:23.773751 20701 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:23.823130 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:23.830391 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.785132 20763 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:23.827481 20948 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:19:23.829177 20944 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:23.820111 20947 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.180s	user 0.002s	sys 0.001s
I20260709 13:19:23.824599 20920 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:19:23.844410 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 95938.1us
W20260709 13:19:23.853055 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 401318us
I20260709 13:19:23.867828 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.867890 20876 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:23.832252 20957 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:23.882277 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:23.842132 20953 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:23.901588 16740 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:19:23.914937 16740 pending_rounds.cc:85] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 5
I20260709 13:19:23.915197 16740 pending_rounds.cc:107] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:19:23.903234 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.904630 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:23.832119 20963 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:23.906697 16559 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:23.845254 20930 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.915452 16740 raft_consensus.cc:2889] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:23.879058 20940 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
I20260709 13:19:23.864269 20920 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:23.915724 16745 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:23.905475 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:23.878444 20946 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 13:19:23.917080 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 224497us
I20260709 13:19:23.750134 20396 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.865411 20701 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.786166 20924 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:23.939258 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.936239 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:23.916270 20957 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:23.930337 20953 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:23.939059 20934 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:23.898190 20876 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:23.961200 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:23.864850 20962 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:23.879755 20735 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 13:19:23.968396 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 138067us
I20260709 13:19:24.010181 20920 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.011313 17374 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10.
I20260709 13:19:24.011847 20920 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:23.976608 20753 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.013438 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.014741 20963 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:24.015166 20963 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:24.015589 20963 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:23.998099 20929 consensus_meta.cc:287] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.003s
I20260709 13:19:23.930832 20919 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.933761 20930 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:23.930812 20968 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:23.999473 20936 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:23.964145 20701 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.028488 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:23.982407 20977 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:23.998404 20790 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.025439 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.036863 20919 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.006131 20928 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:24.056138 17419 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:19:24.056553 17419 pending_rounds.cc:85] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 10
I20260709 13:19:24.056782 17419 pending_rounds.cc:107] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:19:24.057011 17419 raft_consensus.cc:2889] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:24.058375 16592 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:24.031430 20929 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:24.091122 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65974.3us
I20260709 13:19:24.095204 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:24.096796 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:24.097393 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.098453 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:24.048612 20945 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:24.051195 20934 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:24.105522 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:24.048655 20968 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:24.119195 20987 raft_consensus.cc:2764] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:19:24.039271 20977 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.119809 20968 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.125890 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 3106) took 1065 ms. Trace:
I20260709 13:19:24.133894 16588 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:24.121367 20977 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:24.128746 16595 rpcz_store.cc:276] 0709 13:19:23.051633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.051783 (+   150us) service_pool.cc:224] Handling call
0709 13:19:23.052154 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.052899 (+   745us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:19:23.264336 (+211437us) log.cc:844] Serialized 14 byte log entry
0709 13:19:23.877703 (+613367us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:23.937768 (+ 60065us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:23.964853 (+ 27085us) log.cc:844] Serialized 26 byte log entry
0709 13:19:23.965024 (+   171us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:23.965063 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:23.965136 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.106739 (+141603us) raft_consensus.cc:1685] finished
0709 13:19:24.117261 (+ 10522us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.117438 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31987,"prepare.queue_time_us":247906,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":90976,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":34680,"spinlock_wait_cycles":6207744,"thread_start_us":834885,"threads_started":3,"wal-append.queue_time_us":373224}
I20260709 13:19:24.051054 20928 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.151304 20957 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.152786 20790 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:24.153563 16587 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:24.154064 20956 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:19:24.123558 20962 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:24.229812 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 293674us
I20260709 13:19:24.123828 20936 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:19:24.241149 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:24.241844 16595 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:24.244279 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 382694us
W20260709 13:19:24.245080 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 135458us
I20260709 13:19:24.137601 20638 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.127077 20982 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 13:19:24.248679 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:24.251251 16594 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:19:24.241729 20593 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.053s	user 0.004s	sys 0.009s
I20260709 13:19:24.227957 20876 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:24.248299 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:24.282884 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:24.331092 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:24.245352 20969 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.334424 17415 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:24.105786 20953 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:24.337185 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54429.2us
I20260709 13:19:24.253352 20982 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:24.348647 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:24.243585 20670 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:19:24.332273 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:24.282845 20126 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.003s
I20260709 13:19:24.353775 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:24.338326 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:24.243137 20790 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:19:24.354117 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:24.357606 20943 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:24.251101 20920 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:19:24.359603 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:24.346983 20766 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.720s	user 0.011s	sys 0.001s
I20260709 13:19:24.339418 20888 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:24.342830 20972 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:24.366982 16594 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:24.369311 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:24.345921 20930 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.336055 20753 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.355077 20981 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:24.340108 20593 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:19:24.419665 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65648.5us
I20260709 13:19:24.363723 20943 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:24.422253 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 300949us
I20260709 13:19:24.373000 20945 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:24.401093 20766 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11.
I20260709 13:19:24.424348 20929 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:24.355603 20919 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:19:24.358577 20977 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:19:24.427160 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:24.421487 20949 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.412534 20972 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.430173 20928 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:24.363057 20126 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:24.434701 20993 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.080s	user 0.003s	sys 0.000s
I20260709 13:19:24.432775 20945 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.413172 20701 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.444103 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.425740 20983 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.430346 20943 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:24.442180 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:24.436280 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 196735us
I20260709 13:19:24.451395 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:24.349790 20969 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:24.455682 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 208410us
I20260709 13:19:24.447036 20972 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:24.533727 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 67294.7us
I20260709 13:19:24.540934 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.442098 20650 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:24.425590 20753 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.452455 20930 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:24.455013 20945 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:24.559176 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:24.559938 20986 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
W20260709 13:19:24.560570 20989 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:19:24.530694 20766 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:24.580201 16591 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:24.572499 20930 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:24.554554 20650 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:24.580891 16482 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.619s	user 0.002s	sys 0.007s
W20260709 13:19:24.603459 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62644.9us
I20260709 13:19:24.609346 17420 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
W20260709 13:19:24.620754 20937 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.089s	user 0.002s	sys 0.019s
W20260709 13:19:24.577015 21002 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:19:24.643965 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.605293 20766 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:24.451457 20949 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:24.655735 20998 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:19:24.652340 20650 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:24.696931 20920 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:24.698302 20937 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.702999 20982 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.708804 20766 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:24.751204 16637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.020s	sys 0.095s
W20260709 13:19:24.752944 16637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.020s	sys 0.095s
W20260709 13:19:24.747859 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 89324.2us
I20260709 13:19:24.756129 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.757465 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.757526 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:24.759322 17409 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 13:19:24.747972 20357 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.007s	sys 0.017s
I20260709 13:19:24.759994 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:24.760694 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.760953 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:24.765754 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:24.761885 20357 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.007s	sys 0.017s
I20260709 13:19:24.766556 20396 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:24.772408 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:24.773397 21001 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:24.773649 20969 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:24.769217 20763 consensus_meta.cc:287] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.923s	user 0.006s	sys 0.006s
I20260709 13:19:24.775152 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 465) took 1010 ms. Trace:
I20260709 13:19:24.784103 16594 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:24.765837 20934 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:24.774109 21000 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:19:24.787989 17414 rpcz_store.cc:276] 0709 13:19:23.764986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.804684 (+ 39698us) service_pool.cc:224] Handling call
0709 13:19:23.805053 (+   369us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:19:24.235802 (+430749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:24.235834 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:19:24.246767 (+ 10933us) log.cc:844] Serialized 78 byte log entry
0709 13:19:24.338408 (+ 91641us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:24.738343 (+399935us) log.cc:844] Serialized 14 byte log entry
0709 13:19:24.743294 (+  4951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:24.743378 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:24.774927 (+ 31549us) raft_consensus.cc:1685] finished
0709 13:19:24.774948 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:24.775094 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29909,"prepare.queue_time_us":433206,"raft.queue_time_us":112098,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":17427,"thread_start_us":895719,"threads_started":3,"wal-append.queue_time_us":155133}
I20260709 13:19:24.834045 16591 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:24.753167 20982 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.787945 20126 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.755982 20930 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:24.859565 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 87288.2us
I20260709 13:19:24.829011 20934 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:24.876915 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:24.877771 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:24.878360 16664 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:24.859684 21001 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.877090 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:24.871441 20939 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:24.895855 20949 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.896333 20650 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.896770 20934 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.885675 20929 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:24.899389 20929 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:24.900219 21012 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:19:24.902331 20650 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.907377 20766 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.912228 20766 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:24.913014 20929 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:24.921347 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:24.935369 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:24.936589 16665 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:24.882100 21011 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:24.938431 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 61422.8us
I20260709 13:19:24.943130 21006 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:19:24.949158 17409 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 13:19:24.951627 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:24.951951 20955 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:24.952411 20955 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:24.952870 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:24.953627 16670 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:24.954636 16670 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:24.953155 20955 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:24.962379 17415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 466) took 1194 ms (client timeout 1510 ms). Trace:
I20260709 13:19:24.884044 21001 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:24.963392 20650 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:24.964083 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:24.965166 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 468) took 1185 ms. Trace:
W20260709 13:19:24.973313 17415 rpcz_store.cc:269] 0709 13:19:23.768126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.768416 (+   290us) service_pool.cc:224] Handling call
0709 13:19:24.962180 (+1193764us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54004b6254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:24.962318 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298916224}
I20260709 13:19:24.983865 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:24.988104 17412 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:24.981278 16662 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:24.928081 20766 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:24.989657 17420 rpcz_store.cc:276] 0709 13:19:23.779889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.779996 (+   107us) service_pool.cc:224] Handling call
0709 13:19:24.964973 (+1184977us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b540048dfd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:24.965101 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529089920}
I20260709 13:19:24.995165 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:24.937377 20937 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:24.993124 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:24.973730 20963 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.958s	user 0.009s	sys 0.004s
I20260709 13:19:24.888773 20939 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.006230 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:24.972632 21016 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:24.981812 20924 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:24.973050 20983 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:24.988325 20959 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.014376 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.014498 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:24.994823 20650 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.973332 21011 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.970319 20943 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.004068 20766 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.028280 16740 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:25.028469 16744 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:19:25.049062 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 288432us
I20260709 13:19:25.052870 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:25.027818 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.026468 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.027493 16741 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:19:25.022259 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58278.8us
I20260709 13:19:25.049060 20026 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:25.046617 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.075016 16739 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.033491 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:25.055102 20959 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.021648 21015 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:25.030129 20650 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.024036 20638 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.023959 20945 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:24.997876 21006 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:19:25.013386 20753 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.051766 16559 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.074462 16591 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.098661 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.101259 17413 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:25.084523 16669 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:25.083695 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50302.1us
I20260709 13:19:25.103107 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.103456 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:25.084695 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.092557 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:25.084267 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.105154 16593 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:25.087112 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.088166 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.109190 16595 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:25.109259 16669 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:25.110797 16669 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:25.050673 20950 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.045362 20885 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.091810 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.079638 20845 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.136399 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.137193 16592 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:25.019232 20126 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:25.031224 20396 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.116475 16741 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:25.117846 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.030442 20961 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:25.045698 20994 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.034827 20906 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.050446 20953 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.120088 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:25.036845 21001 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.018193 21016 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:25.050225 20957 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.125149 16739 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:25.124293 20889 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:25.091187 20973 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:19:25.126665 16671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1670) took 1625 ms (client timeout 1803 ms). Trace:
W20260709 13:19:25.194835 16671 rpcz_store.cc:269] 0709 13:19:23.501029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.501223 (+   194us) service_pool.cc:224] Handling call
0709 13:19:25.123558 (+1622335us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b540033ac54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:25.126582 (+  3024us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1542366080}
I20260709 13:19:25.089638 20930 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:25.107785 20521 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.269s	user 0.011s	sys 0.002s
I20260709 13:19:25.090018 20938 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:25.086294 20959 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.092078 20933 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.085930 20952 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:25.008204 20963 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.006011 20937 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.020821 20983 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.130635 21004 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:19:25.130777 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 79096.5us
W20260709 13:19:25.131536 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 56077.6us
I20260709 13:19:25.134048 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.087688 20943 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.091481 20987 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:25.089062 20766 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.089968 20638 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.092116 20945 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.152781 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.264056 17420 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:25.153115 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:25.115078 20753 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.154614 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:25.151687 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.157650 17410 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.146660 20885 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.147753 20845 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.081707 20924 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.169914 16591 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:25.170102 16593 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:25.143339 20982 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.149147 20936 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.118750 20969 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:19:25.201514 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:25.212082 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58158.8us
I20260709 13:19:25.135663 20950 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.240571 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:25.242076 16559 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.803s (SENT) [suppressed 47 similar messages]
I20260709 13:19:25.088137 21015 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.192548 20953 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.253052 21016 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.157284 20994 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.223582 20933 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.254501 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 2315) took 1564 ms. Trace:
I20260709 13:19:25.255415 21011 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.196748 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:25.256754 21004 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:25.197395 20889 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.213984 20938 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.259096 20983 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.263032 16587 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:25.435333 16590 rpcz_store.cc:276] 0709 13:19:23.690132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.692177 (+  2045us) service_pool.cc:224] Handling call
0709 13:19:23.716278 (+ 24101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:23.930452 (+214174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:23.930560 (+   108us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:25.051413 (+1120853us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b2c000014c8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:19:25.051811 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:19:25.122370 (+ 70559us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:25.122695 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 13:19:25.135664 (+ 12969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.135762 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:25.254205 (+118443us) raft_consensus.cc:1685] finished
0709 13:19:25.254321 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:25.254442 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45773,"prepare.queue_time_us":168,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":94394,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":23019,"spinlock_wait_cycles":27656448,"thread_start_us":235069,"threads_started":2,"wal-append.queue_time_us":96548}
I20260709 13:19:25.264886 16663 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
W20260709 13:19:25.266832 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 113783us
I20260709 13:19:25.220320 20952 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:25.220291 20961 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:25.277078 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.277217 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.273228 20638 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.271862 20959 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.274044 20753 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.247992 20987 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.295413 20990 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:25.288321 17411 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:25.460394 17415 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:25.430670 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 157803us
I20260709 13:19:25.432500 17410 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:25.439147 16669 raft_consensus.cc:606] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.445676 16742 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:25.447907 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.452454 16744 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:25.435523 20969 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.458829 21018 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.459906 20937 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:25.430243 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 189752us
W20260709 13:19:25.463126 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 221082us
I20260709 13:19:25.439587 20936 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.463822 20987 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.490314 20952 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.487975 21018 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.491431 20952 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:25.492484 20972 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.929s	user 0.017s	sys 0.002s
I20260709 13:19:25.492709 20885 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.492923 20972 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.494074 16588 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:19:25.486948 21034 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.495424 21034 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.497525 21034 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.499181 21022 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.499047 21017 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:19:25.499780 20963 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.500036 21022 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:25.502254 20924 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.494647 20969 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.504081 20889 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.504727 20845 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.506400 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:25.508364 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.513553 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.444641 20396 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:25.498835 21037 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:19:25.514217 16744 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:25.505035 21039 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:25.536615 20026 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:19:25.553260 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.554451 16744 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:25.554535 16665 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:25.554466 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:25.556993 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.557897 16738 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:25.561498 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:25.461499 20906 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.504763 20964 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:25.570292 20972 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.567337 20858 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:19:25.570305 21024 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.223s	user 0.003s	sys 0.006s
I20260709 13:19:25.593950 20858 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:25.594630 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.595886 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.597558 21041 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.598182 21041 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.598680 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.599344 16738 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:25.570544 20974 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:19:25.600128 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.600735 16738 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:25.602607 16738 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:25.603755 20939 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.593623 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.604359 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.604667 16742 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.546809 20987 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.605850 16671 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:25.608554 16744 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:25.566669 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.609822 20987 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.613356 21041 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.573230 21018 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.573275 20933 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.585820 21004 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.587352 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.618053 16670 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:25.583057 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.619407 21041 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.458026 21031 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.504760 20969 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.565826 20929 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.555049 21022 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.568290 20934 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.505398 20961 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.614931 20987 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.581043 20906 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.487241 20753 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.485669 20638 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.504099 20950 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.589959 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.634308 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.590684 16593 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:19:25.591408 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.614886 16667 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:19:25.621419 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.590119 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.588848 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.622190 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.678665 20593 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:25.750533 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 161728us
I20260709 13:19:25.456092 21001 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:19:25.754887 20126 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:25.755348 20126 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:25.678515 17416 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:25.553126 20963 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:25.637866 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 176639us
I20260709 13:19:25.757071 20963 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.757932 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 877) took 1705 ms. Trace:
I20260709 13:19:25.758327 17411 rpcz_store.cc:276] 0709 13:19:24.052935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.059313 (+  6378us) service_pool.cc:224] Handling call
0709 13:19:24.099553 (+ 40240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.464129 (+1364576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.757692 (+293563us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400465354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:25.757876 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2681429248}
W20260709 13:19:25.762899 16663 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1672) took 2226 ms (client timeout 2150 ms). Trace:
W20260709 13:19:25.763347 16663 rpcz_store.cc:269] 0709 13:19:23.536314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.536471 (+   157us) service_pool.cc:224] Handling call
0709 13:19:25.762621 (+2226150us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540009db54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:25.762761 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3782841728}
I20260709 13:19:25.630571 20939 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.764907 20939 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.584868 20994 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:25.581100 21010 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.726s	user 0.005s	sys 0.013s
I20260709 13:19:25.583904 21015 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.589293 21020 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.719120 21009 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.729039 20987 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.772918 21020 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.774049 17413 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:25.776082 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 3138) took 1242 ms. Trace:
I20260709 13:19:25.776657 16588 rpcz_store.cc:276] 0709 13:19:24.534003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.534325 (+   322us) service_pool.cc:224] Handling call
0709 13:19:24.534749 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:25.494449 (+959700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.775724 (+281275us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b5400410854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:25.776005 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2205722368}
I20260709 13:19:25.777578 20963 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.588682 20953 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.463807 20959 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.772970 21009 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.756189 20126 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.782162 20126 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:25.783317 21009 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.778141 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.784195 16588 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:25.786958 20933 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:25.787523 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 470) took 1993 ms. Trace:
I20260709 13:19:25.787676 21035 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:25.787923 17412 rpcz_store.cc:276] 0709 13:19:23.794015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.797739 (+  3724us) service_pool.cc:224] Handling call
0709 13:19:24.968010 (+1170271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:24.990472 (+ 22462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:25.787328 (+796856us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540048dfd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:25.787468 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2689542400}
I20260709 13:19:25.789518 20945 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:25.790047 20945 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:25.790218 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:25.790822 16588 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:25.591082 20701 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.792399 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.792626 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.794691 17413 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:25.714182 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:19:25.795641 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 81529.8us
I20260709 13:19:25.791350 20593 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.796465 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:25.559419 16482 maintenance_manager.cc:643] P bfe27eaae9f4449e96031a0b1485f80c: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 6.955s	user 0.158s	sys 0.063s Metrics: {"bytes_written":29394,"cfile_init":1,"dirs.queue_time_us":29253,"dirs.run_cpu_time_us":521,"dirs.run_wall_time_us":79937,"drs_written":1,"lbm_read_time_us":397,"lbm_reads_lt_1ms":4,"lbm_write_time_us":98290,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":3,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"raft.queue_time_us":138643,"raft.run_cpu_time_us":15524,"raft.run_wall_time_us":235839,"rows_written":63,"thread_start_us":314286,"threads_started":3,"wal-append.queue_time_us":351030}
I20260709 13:19:25.679864 21031 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.720031 16592 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:25.719198 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.797787 20845 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:25.720939 16742 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:25.708155 16669 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:25.798712 20845 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:25.745193 17420 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:25.799360 16595 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:25.748533 16737 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:25.721877 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:25.800077 20936 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.622219 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.800869 17415 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:25.801743 16483 maintenance_manager.cc:419] P bfe27eaae9f4449e96031a0b1485f80c: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8456 bytes on disk
I20260709 13:19:25.803045 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.803236 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.803316 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.804327 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.582854 20938 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.591569 16590 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:25.752526 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:25.793208 16589 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:25.749493 20924 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:25.753641 20670 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:25.619895 21041 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.754628 16634 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.689551 21022 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.744266 17409 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:25.559657 20396 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:25.780905 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 92678.4us
W20260709 13:19:25.629689 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64354.4us
I20260709 13:19:25.820927 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.747586 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.822606 16593 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:25.826119 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.808354 16591 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:25.687592 20929 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.830514 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.789481 20982 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:25.810968 16589 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:25.813396 16592 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:25.818797 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.832980 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.787689 20933 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:19:25.838114 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83554.7us
I20260709 13:19:25.846939 16736 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:25.851249 16482 maintenance_manager.cc:643] P bfe27eaae9f4449e96031a0b1485f80c: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.046s	user 0.002s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":397,"lbm_reads_lt_1ms":4}
I20260709 13:19:25.841678 16559 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.845914 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:25.855554 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:25.807163 20593 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:25.812129 20701 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.860875 20929 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.861691 16668 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:25.845157 20996 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:19:25.859081 21044 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:25.864044 21041 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.864362 21044 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.810206 20670 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.867249 20670 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:25.815714 20945 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.869485 20945 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:25.825475 20924 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.873027 20924 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.696760 20934 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.875914 20934 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.876057 20701 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.641819 20858 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.880686 21022 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.839371 20982 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:25.884680 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:25.885864 16737 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
W20260709 13:19:25.886009 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 44431.3us
I20260709 13:19:25.812017 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:25.889987 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.890780 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.892633 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.894023 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.895179 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:25.896394 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.890821 16669 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:19:25.899355 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:25.899952 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.900996 17409 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:25.901199 16669 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:25.905447 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 93538.1us
I20260709 13:19:25.919824 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.844532 20936 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:25.922758 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.907296 16588 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:25.909727 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:25.926870 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.927822 16588 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:25.891213 21043 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:25.899044 20929 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:25.931205 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.866765 21044 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.931860 16591 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:19:25.897683 20931 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:25.933879 20931 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:25.934242 20931 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:25.934482 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:25.935847 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.937991 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:25.932878 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:25.939502 16737 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.939200 20920 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:25.940675 21018 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:25.941084 21018 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:25.942049 21018 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.945957 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:25.882364 21009 consensus_queue.cc:1175] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 9 status { last_received { term: 9 index: 9 } last_committed_idx: 7 last_received_current_leader { term: 9 index: 9 } }
I20260709 13:19:25.948298 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:25.918385 20631 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:25.949671 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:25.853940 20949 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:19:25.951898 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.918494 20593 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:25.955693 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.955876 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:25.959923 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.908849 21015 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:25.957505 16589 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:25.961197 16737 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:25.918569 20930 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:25.981762 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.965615 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:25.918553 20996 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:19:25.983474 16592 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:25.953495 20952 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:25.930395 21004 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:25.958981 20929 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:25.987174 16591 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:25.962564 21041 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:25.945441 21022 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:19:25.946502 20938 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:19:25.984660 21015 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:25.922565 20982 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:25.981504 20631 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:26.019182 20631 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.002554 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.023837 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:26.024899 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:25.965210 20924 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:19:26.025817 20943 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:26.026364 20963 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:26.026887 20963 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:26.027066 20949 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:26.027616 20949 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:26.028016 20963 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.028448 20949 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.029699 20963 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:26.031638 20889 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:26.032055 20889 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:26.032925 20889 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.034137 20889 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.035079 20952 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.035740 20952 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.036846 20929 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.038000 20952 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:25.998767 21041 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.039136 21041 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:25.988551 20968 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:26.039943 20968 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.040509 20968 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.040038 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.042169 21031 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:25.997077 21018 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.008841 20845 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:19:26.015975 20701 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:25.999017 21004 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.026487 20943 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:26.044580 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.000008 20992 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:25.950273 21043 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:25.987010 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 152205us
W20260709 13:19:26.047610 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.002949 20934 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:25.938849 20670 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:25.998525 20766 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:19:26.015799 21022 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:26.010778 21020 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.050680 20845 consensus_queue.cc:1243] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:26.051575 21004 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:26.073534 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.075938 21047 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:19:26.076706 21045 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:19:26.069533 21052 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.055078 20701 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.106248 20845 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.106546 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.106894 20845 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.115391 20670 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.055168 20992 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:26.084569 20396 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.057711 20982 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:26.085302 21015 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.096961 21041 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.070379 21022 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:26.060565 20943 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:26.120716 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:26.137900 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 188269us
I20260709 13:19:26.140638 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.143353 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.144187 17409 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:26.147063 17413 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:26.119541 20701 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.137411 20964 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:26.165096 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.166131 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.117554 20992 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.103124 21052 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.176465 17409 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:26.176913 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.195741 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:26.177841 21054 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.203089 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.186167 21041 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:26.206980 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 133873us
I20260709 13:19:26.188563 20987 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.194351 20638 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.195848 20992 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:26.193336 20934 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:26.227258 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.228024 16587 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:26.228914 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.227487 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.230006 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.230521 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.197731 20924 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:26.232036 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.234329 16591 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:26.237869 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.241298 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.242785 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.182835 20943 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.238493 16591 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:26.247713 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.248835 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.249073 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.250556 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b778f817264440987b35333e6540c72"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:26.238445 20987 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:26.251261 16669 raft_consensus.cc:606] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.215734 20753 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:19:26.251736 16669 raft_consensus.cc:614] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:26.250406 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:26.252512 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.195961 21022 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.203712 21054 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.241986 20924 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:26.267370 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.267827 20924 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.245280 21041 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.268568 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.269109 21041 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.269819 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.271023 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.271957 21041 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.246094 20983 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:26.275607 21022 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.250710 20938 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:26.280869 21052 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.257681 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.257292 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.259181 20919 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:26.263607 20934 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.265986 21054 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.266600 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.297327 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:26.298599 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.295498 20919 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.302079 21005 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:26.305157 21005 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.305588 21005 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:26.306001 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.306433 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.310472 20931 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.312592 20931 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.299999 21052 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.257372 20987 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:26.326603 20987 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.332801 17409 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:19:26.332835 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.334407 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.336010 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.337556 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.338747 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:26.301651 20919 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:26.354569 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.357052 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:26.339609 20638 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:26.301283 20968 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.383723 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:26.303730 21044 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:26.358060 20889 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:26.357879 20924 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.388756 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:26.356992 21065 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:26.387532 20753 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:26.374388 20953 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.386296 20983 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.329424 20938 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:26.399909 16669 raft_consensus.cc:606] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.411482 16666 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:26.399355 20889 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.424666 17416 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 13:19:26.428857 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 1686) took 1437 ms. Trace:
I20260709 13:19:26.429330 16662 rpcz_store.cc:276] 0709 13:19:24.991545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.994600 (+  3055us) service_pool.cc:224] Handling call
0709 13:19:26.428644 (+1434044us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b540001b654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:26.428794 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874402048}
I20260709 13:19:26.432854 20959 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.400905 21044 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.437083 21044 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.438498 16663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59566
I20260709 13:19:26.439263 16663 raft_consensus.cc:606] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership
I20260709 13:19:26.422562 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.389111 21004 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.440414 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:26.398419 20825 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:19:26.444425 20753 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:19:26.451952 20753 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:26.461022 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59566
I20260709 13:19:26.461844 16669 raft_consensus.cc:606] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership
I20260709 13:19:26.462659 16663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:26.463411 16663 raft_consensus.cc:606] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.463807 16663 raft_consensus.cc:614] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:26.466434 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 128972us
I20260709 13:19:26.467192 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.437919 21035 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:26.468829 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66070us
I20260709 13:19:26.469266 16588 raft_consensus.cc:2368] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:19:26.470597 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.443966 20889 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.447311 21015 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.411865 20953 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.477427 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:26.466337 21019 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.465894 21022 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.468575 20924 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:26.471648 21035 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:26.471395 20992 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.472375 21017 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.489457 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:26.490135 16669 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.489491 16592 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:26.497601 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:26.485651 21022 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.495157 21017 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.509011 21022 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.509902 21017 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.484431 21015 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.516258 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.494006 20992 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.487977 21019 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.520320 21015 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.514945 20968 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.528406 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.534658 20992 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.529199 21004 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.530828 21019 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.531176 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.537853 17416 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
W20260709 13:19:26.544328 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.544938 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:26.514108 21035 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.546603 20670 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.551133 20396 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:26.552634 20939 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:26.553916 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:26.558126 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.547716 20670 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.560158 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:26.562465 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.562793 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.563091 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.563915 16669 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:26.564342 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.564631 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.566987 20929 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:26.563385 16665 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:26.567721 20929 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.568161 20929 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.569254 20919 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.569705 20906 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.569877 21005 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.570639 20845 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.571943 20953 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.576613 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.577121 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.578114 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.579051 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.580104 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.582645 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.578836 21034 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.583935 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.584136 21034 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.580520 16592 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:26.587761 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:26.548259 21035 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:26.589272 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.589994 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.590227 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.590401 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.590798 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.591171 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.590623 16669 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:26.592815 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.595090 20930 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:26.596295 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.579295 20964 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:26.601038 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:59566
I20260709 13:19:26.601701 16665 raft_consensus.cc:606] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership
I20260709 13:19:26.593669 20396 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:19:26.591710 20939 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:26.595293 20670 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.586107 21029 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.598696 20766 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:26.597309 21018 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.599061 20906 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:26.579353 20950 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:19:26.588990 20953 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:19:26.598840 21033 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.619007 21033 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.619623 21043 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.601548 20964 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.591815 21017 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:19:26.621297 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.613600 20766 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.616884 20919 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.625396 21033 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.627791 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.622062 20670 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.630903 20670 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.633073 20670 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.615417 20906 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.635349 20906 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:26.616611 21018 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.620869 21034 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.637398 21044 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.628388 20766 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.638568 16743 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:19:26.639029 21018 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:26.643268 21074 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:19:26.626890 20964 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:26.628827 20858 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.613724 21029 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.661520 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 455) took 1785 ms. Trace:
I20260709 13:19:26.660593 21044 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.662189 21029 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.662743 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:26.661954 16667 rpcz_store.cc:276] 0709 13:19:24.875690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:24.875832 (+   142us) service_pool.cc:224] Handling call
0709 13:19:24.876081 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.744399 (+868318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.643078 (+898679us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b540033b154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:26.661466 (+ 18388us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698453760}
I20260709 13:19:26.663590 20977 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:26.664007 20977 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.664389 20977 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:26.665199 20889 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.667716 21005 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.668615 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:26.669356 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:26.677915 20977 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.680835 16596 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:26.683512 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.703567 16595 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:26.700306 20939 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:19:26.705158 20939 consensus_queue.cc:1243] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:26.707006 20977 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.711463 20955 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.683741 20766 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.713553 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1693) took 1156 ms. Trace:
I20260709 13:19:26.713986 16668 rpcz_store.cc:276] 0709 13:19:25.556888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.596110 (+ 39222us) service_pool.cc:224] Handling call
0709 13:19:25.596439 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:25.862081 (+265642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:26.713298 (+851217us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540009e054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:26.713487 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":609180416}
I20260709 13:19:26.715201 20766 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.717309 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.718418 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.719803 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.720139 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:26.702099 20858 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.720852 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.721061 20858 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.721820 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.723796 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:26.724555 21020 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:26.725018 21020 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:26.725777 21020 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.726456 16742 pending_rounds.cc:85] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 6
I20260709 13:19:26.726737 16742 pending_rounds.cc:107] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:19:26.726985 21036 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.727207 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:26.727384 16745 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:26.727532 21036 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.728049 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.728856 16740 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:26.729611 21036 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:26.730362 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.684945 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.731462 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:26.731757 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.732100 16741 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:26.732769 20963 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:26.678711 20964 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.726992 16742 raft_consensus.cc:2889] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:26.734992 20396 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:26.735119 20026 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:26.735800 16587 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:26.735539 20396 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:26.736825 16665 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:26.737571 20396 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.737274 20964 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.731510 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.739006 16593 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:26.739657 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.739990 20026 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.741255 16667 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:26.696131 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.697077 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.699821 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.743191 16589 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.611016 20929 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:26.731581 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46730.5us
I20260709 13:19:26.748256 16745 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:26.741217 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.754150 16592 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:19:26.755167 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 86615.7us
I20260709 13:19:26.756315 16663 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:26.757936 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.758436 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.758687 20026 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:26.761745 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:26.760775 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:26.763350 16665 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:26.767508 16669 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:19:26.767890 16669 pending_rounds.cc:85] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 8
I20260709 13:19:26.768095 16669 pending_rounds.cc:107] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:19:26.768330 16669 raft_consensus.cc:2889] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:26.769222 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:26.770560 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.771270 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.771484 16588 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:26.773224 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:26.754911 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.775409 16592 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:26.759490 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:26.780094 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:26.781013 16743 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.786127 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.786813 16663 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:26.788038 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:26.788915 16739 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:19:26.790634 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:26.791656 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.792865 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.794983 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.795637 16745 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:26.795854 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.799924 16667 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:26.804831 16745 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:26.734769 21017 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.774034 20953 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:26.806685 16745 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:26.810895 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:26.817272 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.818801 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:26.785326 20766 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.821585 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.778504 20924 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:26.822465 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:26.839051 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.825022 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.840595 16663 raft_consensus.cc:2410] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:19:26.784708 20977 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.826661 16708 leader_election.cc:400] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:19:26.841173 16708 leader_election.cc:403] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:26.829665 16589 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:26.829944 16743 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:26.838001 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.797408 21035 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:26.846200 16745 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:26.847671 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.796152 20987 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:26.850286 16669 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:26.765614 20963 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:19:26.852238 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.853786 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.841976 21043 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:26.825811 21033 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:26.849565 21035 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.877318 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.857990 16739 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:19:26.820777 21015 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:26.838546 21041 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:26.871832 16663 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:19:26.880465 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.905750 16670 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:26.860951 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.847970 21031 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:26.881587 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:26.893059 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:26.898463 16741 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:26.862216 20959 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:19:26.886389 20858 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:26.889750 20987 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:26.846632 21022 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:19:26.820780 21017 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:26.910257 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 87917.1us
I20260709 13:19:26.935456 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:26.936247 17412 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:26.903906 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:26.863318 20977 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:26.928289 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46253.8us
I20260709 13:19:26.890662 21033 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:26.945570 21033 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.902542 21028 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:19:26.941823 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:26.908147 21041 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:26.943222 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:26.971987 16592 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:26.909969 21031 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:26.903568 21035 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:26.979713 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:26.895318 20992 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:19:26.932906 20396 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:27.034809 16670 pending_rounds.cc:85] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 9
I20260709 13:19:27.036001 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.039208 16670 pending_rounds.cc:107] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:19:27.039829 16670 raft_consensus.cc:2889] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:27.037536 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57954.4us
I20260709 13:19:26.908437 21015 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.046339 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.049103 16587 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:26.967669 20858 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.051923 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:26.975523 21041 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:27.043325 20766 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.099649 20766 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.100306 20929 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.042816 21036 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:27.102886 20766 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.046728 20701 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.103749 20701 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.104177 20945 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:27.106514 20701 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.073424 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.107414 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:27.107697 20701 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:27.108325 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.108590 20953 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:27.075779 20858 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.109531 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.075294 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.111294 20858 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.076843 21017 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:26.942009 21019 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:19:27.081403 20971 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:27.112843 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.113233 20971 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:27.113638 16671 raft_consensus.cc:2410] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:19:27.082442 20950 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.115200 20930 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:27.115013 20950 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.115708 20930 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.115962 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:27.120209 16594 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:27.121982 20950 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.089157 16665 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:19:27.090847 20992 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:27.092311 16741 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:27.093745 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.127903 20992 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:26.974093 21031 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:27.077860 21077 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:19:27.096866 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.097069 20906 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.043010 20396 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.104331 21015 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.066162 17412 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:27.066269 20977 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.108747 20701 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:27.112221 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.113618 20971 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:27.082154 20943 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:27.085707 16739 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [8.6-11.8]
I20260709 13:19:27.094274 20889 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.156337 16741 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:27.156786 17414 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:27.095585 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:27.171463 16594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 2321) took 2076 ms (client timeout 1707 ms). Trace:
W20260709 13:19:27.172335 16594 rpcz_store.cc:269] 0709 13:19:25.074546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.074686 (+   140us) service_pool.cc:224] Handling call
0709 13:19:27.150684 (+2075998us) spinlock_profiling.cc:243] Waited 2.01 s on lock 0x7b54000514d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:27.150792 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4626187136}
I20260709 13:19:27.143177 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:27.145887 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.146910 16592 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:27.136276 21065 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:27.185372 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:27.187322 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 276649us
I20260709 13:19:27.145051 20396 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.193107 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.193681 16741 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:19:27.194216 16741 pending_rounds.cc:85] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 10
I20260709 13:19:27.195451 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.196108 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.198460 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:27.198961 16741 pending_rounds.cc:107] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:19:27.199332 16741 raft_consensus.cc:2889] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:27.196388 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.196688 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:27.200199 16745 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:27.197528 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.201056 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.201290 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:27.201488 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:27.189469 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46383.7us
I20260709 13:19:27.202368 17419 raft_consensus.cc:2410] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:19:27.200884 16665 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:27.202904 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:27.205299 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.212996 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.202757 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.214675 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.158335 20906 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.221697 20026 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:27.213047 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:27.170166 21017 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.230675 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:27.225870 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:27.168298 20701 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:27.221992 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:27.238725 20026 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:27.239768 20026 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:27.242587 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.243795 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.262367 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.150266 21020 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:19:27.187095 21065 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:27.267964 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:27.268999 16671 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:27.275785 21034 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:27.278564 20933 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.235385 20766 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.224674 20638 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:27.279918 20638 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:27.280332 20638 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:27.221091 20889 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.287864 16745 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:27.288905 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.289942 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.291175 20825 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:27.291639 20825 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.293432 20825 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.293990 20971 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.295192 20971 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.274353 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:27.279142 20933 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.297371 20987 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:27.297824 20933 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.298271 17409 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:19:27.225224 20963 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:19:27.227104 20945 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:27.301960 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.301795 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.219894 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:27.229893 21036 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:27.309890 20929 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:27.310622 20929 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:27.311223 20933 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:27.311782 20929 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.312904 20889 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:27.313287 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.314039 20938 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:27.314301 21062 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.314424 20938 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:27.314672 20943 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:27.315171 20943 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.315554 20943 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:27.315485 20938 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.316953 20938 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:27.320559 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.321270 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.321779 21062 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:19:27.324687 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.325496 21028 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.327565 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.334095 21018 raft_consensus.cc:993] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:27.337455 20933 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:27.338973 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.339807 20933 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:27.343192 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:27.344707 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:27.307525 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 87716.1us
I20260709 13:19:27.334743 21018 raft_consensus.cc:1081] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:27.300197 21029 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.396215 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.266531 21017 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.348596 20945 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:27.407769 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.408855 16739 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:19:27.389325 21061 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.028s	sys 0.035s
I20260709 13:19:27.417243 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.418047 16744 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:27.399611 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.303968 21086 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
W20260709 13:19:27.423183 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57088.3us
I20260709 13:19:27.444248 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.444666 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.444993 16664 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
W20260709 13:19:27.419885 21061 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.028s	sys 0.036s
I20260709 13:19:27.458813 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.460362 16742 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:27.461506 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.463739 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.465368 16742 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:27.469157 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.471724 20982 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:27.400373 21015 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:19:27.476555 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:27.400738 21033 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:27.475915 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 249701us
I20260709 13:19:27.432840 20889 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:19:27.475598 21043 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.001s
I20260709 13:19:27.493770 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.493643 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:27.499548 16587 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:27.493973 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:27.500706 16593 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:27.476950 16594 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:27.501786 16594 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:27.502244 16594 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.459458 21018 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.504575 21018 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.510349 21088 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.510934 21088 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.513089 21088 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.404430 21065 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.520603 21041 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.513329 21018 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.522001 21018 raft_consensus.cc:993] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:27.522387 21018 raft_consensus.cc:1081] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:27.525282 16594 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.502321 21043 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:27.528190 21033 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.528384 20992 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:27.490949 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46385.5us
I20260709 13:19:27.534819 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:27.536247 20964 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:27.537678 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:27.538411 21033 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:27.536202 21041 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.537806 16594 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.542239 21033 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:27.545006 21033 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.546196 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.548413 20943 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.551419 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.552248 16737 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:27.556133 16744 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:27.557654 16742 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:27.542791 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:27.567381 21015 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:27.499812 20945 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.567941 21015 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:27.568326 21035 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.525s	user 0.007s	sys 0.001s
I20260709 13:19:27.568836 21035 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.569124 21015 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.571358 16738 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:19:27.550655 20943 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:27.571332 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 358339us
I20260709 13:19:27.576907 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:27.570715 21015 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:27.498974 20930 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.588533 20026 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:27.595201 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:27.610335 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:27.582000 20858 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.613829 16669 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:27.613969 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.587867 21017 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.606447 21020 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:27.619621 21020 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:27.620096 21020 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:27.607280 20943 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:27.639871 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 211053us
I20260709 13:19:27.642709 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.642915 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.643424 16742 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:27.643796 16744 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:27.650574 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.651342 16742 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:27.637981 20992 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.655011 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:27.641913 20964 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:27.661234 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.667378 16737 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:27.670248 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:27.671191 20992 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:27.673434 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.640614 21099 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:27.670286 21035 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.658447 20950 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.680356 20950 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.660609 20858 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:19:27.684489 20858 raft_consensus.cc:2764] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:19:27.685246 20858 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:27.685648 20858 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.686033 20858 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:27.691534 21100 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:27.641739 20396 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:27.702795 21098 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:19:27.678779 21031 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 13:19:27.704691 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:27.709215 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70882.3us
I20260709 13:19:27.659452 20971 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:19:27.712929 16664 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:27.660867 20953 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:19:27.717878 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:27.645308 20638 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:27.714744 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.715124 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.715337 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.727114 16671 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:27.710561 16668 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:19:27.678593 21099 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:27.722791 16664 raft_consensus.cc:2410] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:27.697190 20950 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.724236 16665 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:27.760166 16542 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:19:27.760622 16542 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for fc0ed8d0a60640a0a5ce3a772c3eef5c already in progress
W20260709 13:19:27.760888 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for fc0ed8d0a60640a0a5ce3a772c3eef5c already in progress
I20260709 13:19:27.761205 20753 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:27.761662 20753 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:27.750417 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.763469 16664 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:27.710493 21031 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.745271 21099 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:27.765638 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.766031 21031 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.772363 16745 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:27.759685 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:27.744495 16668 pending_rounds.cc:85] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 7
I20260709 13:19:27.778821 16668 pending_rounds.cc:107] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:19:27.779109 16668 raft_consensus.cc:2889] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:27.753819 20971 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:27.760088 20396 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.711784 21035 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:27.721899 20992 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.745704 16667 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:27.788290 16667 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:27.788064 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:27.798678 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 128500us
W20260709 13:19:27.799583 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.789562 16667 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.800657 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:27.801072 16667 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.781706 21035 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:27.814004 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:27.816439 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.818816 16737 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.788582 21018 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.826828 16741 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:27.826761 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:27.834245 16667 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:27.799450 20945 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:19:27.837347 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:27.837915 16744 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:27.835186 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:27.841118 16739 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:19:27.841852 16737 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:27.804741 20953 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:19:27.847227 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:27.803009 21017 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:27.850565 16671 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:27.851408 16744 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:27.852659 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:27.856393 20701 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.858072 20701 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:27.826593 21035 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:27.830001 20950 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:27.762737 20753 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:27.903748 21108 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:27.848242 21062 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:27.920897 21062 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:27.921271 21062 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.839277 21018 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.899755 21017 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:27.924888 21018 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.925019 21017 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.836604 21101 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:27.928608 21091 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.005s
I20260709 13:19:27.944196 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:27.935250 20753 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:27.842742 21115 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:27.934973 20825 consensus_queue.cc:1175] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" responder_term: 11 status { last_received { term: 11 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 11 index: 9 } }
I20260709 13:19:27.948168 20026 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:27.950263 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:27.955564 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:27.957947 20026 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:27.958443 20026 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:27.959233 20026 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.931979 21101 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:27.964260 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:27.947552 20950 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:19:27.968353 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:27.972122 20924 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:27.994167 20924 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:27.974069 20638 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:27.950070 21115 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:27.981853 20963 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:27.995954 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:27.997262 21115 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.979674 20987 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:27.976636 21101 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.963840 21119 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:27.964980 21099 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:27.979491 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 705193us
I20260709 13:19:28.022734 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.024684 20766 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:28.024956 21099 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:27.995978 20963 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.015101 21062 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:28.026338 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.026908 20766 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.026973 20963 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.024971 21119 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:28.027555 20766 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.028672 21119 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.029645 20766 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.029904 21020 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:28.022404 21081 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:19:28.022557 21121 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 13:19:27.982460 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 492803us
I20260709 13:19:28.032527 21065 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:28.027135 21062 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:27.985147 17414 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:27.978814 20858 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:28.047789 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 79562.3us
I20260709 13:19:28.048697 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:28.051405 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:28.009392 20955 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:28.059760 17415 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:28.010828 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2916) took 1170 ms. Trace:
I20260709 13:19:28.013123 20930 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.013633 21035 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.065754 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.066023 21035 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.017547 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.066426 16739 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:28.068507 16741 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:28.069761 16741 pending_rounds.cc:85] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 9
I20260709 13:19:28.070032 16741 pending_rounds.cc:107] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:19:28.070281 16741 raft_consensus.cc:2889] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:28.071218 16743 raft_consensus.cc:2410] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:19:28.019131 16737 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:19:27.985657 17409 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:28.072446 20858 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.042295 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:28.074059 17410 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:28.074354 17413 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:28.060947 16738 rpcz_store.cc:276] 0709 13:19:26.840199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.840990 (+   791us) service_pool.cc:224] Handling call
0709 13:19:26.867428 (+ 26438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.571878 (+704450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.010518 (+438640us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54002b93d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:28.010728 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1618318720}
I20260709 13:19:28.062300 21043 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:19:28.079828 20977 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:28.083280 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:28.089470 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:28.096745 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:28.097431 16737 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.060218 20955 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:28.098346 20955 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.096819 20858 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:19:28.104295 21043 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:28.104808 21043 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:28.105716 21043 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.088205 21126 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:28.114521 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:28.137511 16737 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
W20260709 13:19:28.163146 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 79955.2us
I20260709 13:19:28.087935 21035 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.172202 16592 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:28.172004 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.174939 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.182433 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.178835 20906 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:28.190088 20906 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.190503 20906 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.192900 16590 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:28.195428 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:28.196205 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.196386 16591 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:28.200055 16594 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:28.169652 21043 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:28.187947 20955 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:28.213023 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.182013 21029 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:28.113729 20701 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 13:19:28.179421 20963 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:19:28.168666 20889 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:19:28.227094 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.211347 20987 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:19:28.188504 21035 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.244542 21035 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.245988 16739 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:28.249408 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:28.250502 20889 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:28.250942 20889 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:28.218586 20996 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:19:28.251744 20889 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.252982 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:28.256151 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.256717 20889 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.266680 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.268215 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:28.268915 16593 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:28.269349 16593 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.271441 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.272294 16592 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:28.276221 16596 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:28.277520 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.281013 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.281867 16596 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:28.283520 16594 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:28.216841 21020 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:28.292874 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.290712 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83118.3us
I20260709 13:19:28.294770 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:28.250991 20963 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:28.297158 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:28.300603 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51277.9us
I20260709 13:19:28.300721 16590 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:19:28.250550 20858 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.246591 21029 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:28.299230 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46319.5us
W20260709 13:19:28.309401 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53294.3us
I20260709 13:19:28.310096 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:28.310956 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.310750 16589 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:28.311434 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.294356 21101 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.297924 20939 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:28.322342 16587 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:28.325348 16590 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:28.326678 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:28.314628 16589 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:28.330577 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:28.310487 16593 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.300292 20963 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:28.360802 20963 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.362049 20963 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:28.329564 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.331862 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:28.335121 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.336400 21029 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.341482 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:28.329310 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:28.352232 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 486) took 1417 ms. Trace:
I20260709 13:19:28.354003 20906 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.311652 20858 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.355672 21101 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.356922 20939 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.313848 20945 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:28.373291 16589 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:28.307833 21061 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:28.406095 20929 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:28.409880 16589 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:19:28.416154 16590 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:28.381284 17411 rpcz_store.cc:276] 0709 13:19:26.934420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.935050 (+   630us) service_pool.cc:224] Handling call
0709 13:19:26.955748 (+ 20698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:27.133671 (+177923us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:27.133704 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:27.134208 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:19:27.433115 (+298907us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:27.473791 (+ 40676us) log.cc:844] Serialized 14 byte log entry
0709 13:19:27.982018 (+508227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:28.351848 (+369830us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540030cd54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:28.351902 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:28.351944 (+    42us) raft_consensus.cc:1685] finished
0709 13:19:28.352031 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:28.352173 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":507994,"prepare.queue_time_us":40686,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":5878,"raft.queue_time_us":512703,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":11971,"spinlock_wait_cycles":408175872,"thread_start_us":298520,"threads_started":1,"wal-append.queue_time_us":368717}
I20260709 13:19:28.436419 16664 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:19:28.409554 16593 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.411007 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:28.447110 16669 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:28.447486 16669 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.448304 16669 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.448765 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:28.449384 16669 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.411878 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:28.414011 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.450799 17410 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.450659 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:28.400552 21022 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:28.454614 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.458530 20026 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:19:28.463296 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.463996 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 167884us
W20260709 13:19:28.464098 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.464684 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:28.442392 21061 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:28.447103 20945 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:28.469766 17410 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:28.449887 21054 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:28.460964 21103 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:28.458163 21022 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.460206 21041 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
W20260709 13:19:28.469765 21138 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:19:28.463152 21015 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:28.468874 20930 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:28.488471 20945 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.501247 20753 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:28.501950 20930 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.578725 20930 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:28.485358 21015 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.583293 21015 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:19:28.585845 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.586027 16489 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 142) took 17653 ms (client timeout 15000 ms). Trace:
I20260709 13:19:27.723145 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:28.588025 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.484355 21022 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:28.536264 21061 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.473239 20919 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:19:28.471566 20825 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:28.485754 21036 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:28.621305 21036 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.562332 20858 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.562901 20906 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.564181 20939 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.564852 21041 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.456940 21088 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:19:28.471721 21017 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:19:28.567869 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.570055 16664 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:28.564054 21101 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:28.571205 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 317908us
I20260709 13:19:28.574833 21020 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.575558 16595 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:28.576771 16669 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:28.578104 20766 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:28.481528 21054 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.661378 21054 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.593288 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:28.594445 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 136315us
I20260709 13:19:28.664016 16736 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:28.601552 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.601572 16745 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:28.665042 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:28.665751 16744 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.634856 20906 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:28.666579 20906 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.667366 21019 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:28.668371 21019 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:28.668725 20906 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.669572 21020 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.671803 21020 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.672103 20955 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:28.672672 20955 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.678766 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.630977 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.636554 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:28.640971 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:28.649739 16738 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:19:28.656333 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.682312 16662 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:28.656275 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.665665 16636 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:28.682654 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:28.683846 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.683873 16663 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:19:28.684276 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:28.685384 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54805.9us
I20260709 13:19:28.686154 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.688242 16739 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:28.633517 21115 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:28.689759 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.691076 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.693181 20026 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:19:28.692184 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.696054 16744 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:28.702337 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:28.705677 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52377.3us
I20260709 13:19:28.706472 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:28.674644 20955 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.713164 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.707773 16667 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:19:28.649075 20766 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:28.637270 20825 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.644778 21140 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.723942 20026 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:19:28.689042 21103 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:28.733196 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:28.723222 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:28.727787 20766 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.702363 20964 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:28.721118 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.668725 21019 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.705585 20919 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:28.736738 20026 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:28.739007 20026 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.717412 21107 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:28.700115 21115 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.722841 21088 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:28.740796 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:28.640796 21041 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.732386 20889 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:28.748498 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.749840 17412 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:28.734592 20638 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:28.734827 21092 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.707688 21118 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:19:28.754010 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:28.742127 21043 consensus_meta.cc:287] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260709 13:19:28.743171 20026 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:28.759467 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.732339 21105 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:19:28.764482 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44222us
I20260709 13:19:28.739897 20701 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:19:28.745467 21103 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:28.744299 20766 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.776790 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.746171 20977 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:19:28.763377 21088 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:28.798707 21155 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:19:28.799671 21019 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.802977 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.807229 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.808132 16593 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:28.778249 20026 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.760169 20889 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:28.744082 21020 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:28.780951 21115 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.777186 21036 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.778481 21103 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.767012 20638 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:28.737676 21140 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.762444 21092 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.790164 21022 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.793738 21041 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:28.586450 16489 rpcz_store.cc:269] 0709 13:19:10.932315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:10.932914 (+   599us) service_pool.cc:224] Handling call
0709 13:19:28.585919 (+17653005us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:24.018556 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:24.037931 (+ 19375us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:24.037987 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:19:24.038010 (+    23us) tablet.cc:662] Decoding operations
0709 13:19:24.111713 (+ 73703us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:24.111815 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:24.111856 (+    41us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:19:24.113918 (+  2062us) tablet.cc:701] Row locks acquired
0709 13:19:24.113983 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:19:24.114212 (+   229us) write_op.cc:270] Start()
0709 13:19:24.114348 (+   136us) write_op.cc:276] Timestamp: P: 1783603164114132 usec, L: 0
0709 13:19:24.114369 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:19:24.115456 (+  1087us) log.cc:844] Serialized 7709 byte log entry
0709 13:19:24.962992 (+847536us) op_driver.cc:464] REPLICATION: finished
0709 13:19:25.243936 (+280944us) write_op.cc:301] APPLY: starting
0709 13:19:25.256129 (+ 12193us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:25.257826 (+  1697us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:25.257970 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:25.695590 (+437620us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:25.709856 (+ 14266us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=19,key_file_lookups=19,delta_file_lookups=0,mrs_lookups=38
0709 13:19:25.709921 (+    65us) write_op.cc:312] APPLY: finished
0709 13:19:25.760240 (+ 50319us) log.cc:844] Serialized 248 byte log entry
0709 13:19:25.760483 (+   243us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:25.760693 (+   210us) write_op.cc:454] Released schema lock
0709 13:19:25.760786 (+    93us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:25.761199 (+   413us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:25.761235 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1308917,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":241107,"apply.run_cpu_time_us":31263,"apply.run_wall_time_us":523452,"cfile_cache_hit":36,"cfile_cache_hit_bytes":37656,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6219,"cfile_init":1,"lbm_read_time_us":988,"lbm_reads_lt_1ms":8,"num_ops":19,"prepare.queue_time_us":254342,"prepare.run_cpu_time_us":7859,"prepare.run_wall_time_us":97327,"replication_time_us":804994,"spinlock_wait_cycles":62720,"thread_start_us":281604,"threads_started":2}]]}
I20260709 13:19:28.796382 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.797595 20950 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.810149 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.775241 20964 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:28.837775 20950 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.838193 20964 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:28.839315 20964 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.840108 20950 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.840699 20950 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:28.841192 20950 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.812525 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:28.818619 20889 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.800920 21019 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:28.846777 21041 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:28.847390 21041 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.848814 20753 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:28.850570 21041 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.854962 17415 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:19:28.858018 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:28.859418 20950 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.826300 20638 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.861060 21117 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:28.828778 21092 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.861492 21117 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.830314 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.862097 16744 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:28.863412 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:28.831075 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.831543 21022 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.832062 20825 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.832808 20930 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.833607 16740 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:28.834750 21140 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.816550 21107 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:28.878463 21103 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.879446 20930 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:28.850955 21036 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:28.852429 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:28.854197 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.893781 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.894392 16589 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:28.894994 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.895767 16668 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:28.896317 21020 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.896842 21020 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.897719 16591 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:28.899210 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.899760 16593 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:28.902015 16589 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:28.902144 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:28.903379 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.903986 16662 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:28.859766 20858 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:28.857975 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:28.812045 21062 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:28.905004 21062 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:28.906611 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96576.8us
I20260709 13:19:28.907130 21062 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.907321 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.907573 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:28.862368 21117 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.816617 21015 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.908851 20930 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:28.909499 21107 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:28.910140 21107 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:28.864883 21092 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:28.835917 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:28.888878 21156 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:19:28.907044 21105 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:28.820389 20026 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:28.911729 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53845.2us
I20260709 13:19:28.913223 16593 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:28.914580 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:28.921766 16587 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:28.925490 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.926484 16663 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:28.914938 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:28.917300 21107 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.935662 21107 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:28.909999 21015 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.917814 20929 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:28.942494 20929 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.943404 20929 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.944571 20929 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:28.918076 21092 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.947086 21092 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:28.911221 17409 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:28.918536 21062 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:28.919046 21106 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:28.951835 21106 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.908483 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:28.951766 21062 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:28.957124 21062 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:28.971933 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56671.7us
I20260709 13:19:28.972486 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:28.973369 16744 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:19:28.810875 21159 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.110s
W20260709 13:19:28.975666 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53994.1us
I20260709 13:19:28.979377 21020 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:28.981961 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:28.982997 21029 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:28.983465 21029 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.985539 21029 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:28.986779 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:28.988922 21114 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:28.990787 20026 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:28.991142 20026 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:28.990696 21114 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:28.992309 21114 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.993315 21106 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:28.994362 21106 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.991431 20026 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:28.998143 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:28.998948 16670 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:29.000789 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:29.002584 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 94171us
I20260709 13:19:29.004405 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.007055 16587 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:19:29.008248 16595 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:29.008329 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.008919 16669 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:29.028085 20026 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.012826 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:29.028662 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 113788us
I20260709 13:19:29.031822 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:29.035429 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.036811 20026 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.042838 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.043504 16590 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:19:29.046157 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.047245 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:29.047547 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.048357 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:29.044840 20638 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:29.045849 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:29.051374 16591 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:29.055639 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.058109 20931 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:29.061215 20977 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:29.061793 20977 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.012107 21164 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:29.033025 20971 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:29.063323 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:29.080772 17412 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:29.065722 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.057359 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.083313 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.085572 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.086226 16739 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:29.055652 21158 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:29.089022 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:29.090551 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.091475 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.081867 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.095940 16743 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.096741 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:29.097913 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.100463 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.046185 21147 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.101660 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.107940 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:29.109018 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.111125 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:29.113428 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.112752 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.117640 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:29.062896 20996 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:29.106163 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:29.062706 20919 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:29.044569 21160 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:29.118544 17409 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:29.121039 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.121717 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.073555 20858 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:29.077613 21101 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:29.094112 20971 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:29.124511 21164 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:29.106654 21159 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:29.127244 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.135246 16738 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:19:29.131462 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.106412 21158 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:29.105803 21103 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:29.076417 21028 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.107076 21105 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:29.121560 21035 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:29.121296 20931 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:29.080369 21062 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.130781 21101 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.135133 20971 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:29.139325 21159 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:29.144856 20825 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:29.144958 21029 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:29.121871 21147 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.144744 21164 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:29.145046 21105 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:29.145432 21103 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:29.172535 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.173174 16666 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:29.161813 16739 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:29.133024 20858 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.174896 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:29.175556 20858 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.120782 21107 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:29.178624 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60790.6us
I20260709 13:19:29.181979 21051 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:29.182476 21051 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.181991 20753 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:19:29.185889 20933 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:29.161398 21101 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.187026 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.166571 16743 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:29.191071 16745 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:29.166487 21159 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:29.194339 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.155402 21158 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:29.186300 20933 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:29.196811 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:29.165596 20825 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:29.194777 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.209792 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.158752 21028 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.156486 21035 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.189170 21105 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.213315 17412 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:19:29.213888 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.219906 17412 pending_rounds.cc:85] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 8
I20260709 13:19:29.189021 21103 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:29.220196 17412 pending_rounds.cc:107] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:19:29.220450 17412 raft_consensus.cc:2889] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.223294 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:29.224099 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.225447 16671 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:29.225529 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.226188 21107 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.191566 20929 raft_consensus.cc:2764] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:19:29.227995 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.234714 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.236101 16636 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.229067 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.229296 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.230664 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.231642 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.245500 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:29.245990 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.232872 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.211885 20825 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:29.188555 21029 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.249924 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.249272 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:29.226547 21166 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:19:29.251492 16744 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:29.268225 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:29.271271 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 91357.6us
I20260709 13:19:29.272276 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.252600 21041 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:29.245157 21094 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:29.275243 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.275838 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.278386 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.278506 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.278648 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.267817 20977 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.268806 20766 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:29.230327 21114 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:19:29.284228 21158 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.294409 21114 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:29.294765 21114 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:29.295650 21114 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.276855 21036 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:29.285925 20963 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:29.286463 21171 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:19:29.287106 21175 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:19:29.288354 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.251907 21107 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.289309 21147 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.297024 21114 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:29.280169 21041 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:29.280468 21043 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:29.289433 21172 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:19:29.277570 21092 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.328740 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.329972 21092 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.330209 16740 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:19:29.281158 20933 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:29.290463 21170 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:19:29.253782 21015 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:19:29.332509 21105 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.341958 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.342643 16740 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.342779 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:29.343705 17412 raft_consensus.cc:2410] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:19:29.290033 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.313936 16736 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:29.315667 21028 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.271908 21164 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.320174 21035 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.320572 21107 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.321231 21062 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.331442 20933 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:29.353075 21159 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.354240 16671 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
W20260709 13:19:29.354336 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64403.8us
I20260709 13:19:29.355080 21028 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.355613 21028 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.356721 21103 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.357507 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.357547 21028 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.357510 16740 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:29.359253 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:29.360308 16662 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:29.361604 20996 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:29.357895 21092 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:29.347795 16741 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:19:29.345978 20971 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.365525 21092 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:29.347102 20825 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:29.370520 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.372750 21101 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.334729 21105 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.373471 20825 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.371228 21159 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.378203 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.391902 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:29.392596 17415 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 8.
I20260709 13:19:29.394517 20971 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.397210 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:29.397209 16744 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:29.388069 21103 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.398677 16744 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:29.362807 20996 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:29.402093 20996 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.403959 21115 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.399729 16744 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.405889 16744 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.400705 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:29.407646 21107 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:29.408077 21107 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.409849 21107 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.415313 21101 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.428210 21107 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.428712 16744 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:19:29.429190 16744 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 6
I20260709 13:19:29.429440 16744 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:19:29.429697 16744 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.430658 21107 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.431584 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:29.432166 17412 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.433717 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:29.434320 16593 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:29.407292 21092 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:29.435925 21092 consensus_queue.cc:1243] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:29.436451 21092 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:29.436760 21092 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:29.437608 21092 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.441663 21092 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:29.439581 17412 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:29.448742 21041 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.449591 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:29.449380 21041 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.453065 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:29.454110 16745 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:29.453521 16593 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:29.455699 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:29.454519 21041 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.456430 20766 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:29.456951 20766 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:29.457357 20766 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.457536 21107 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:29.458067 21107 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:29.458825 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.459067 21107 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.464828 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.465448 16668 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:29.475519 21029 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.494233 20766 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.500229 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.500784 16668 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.502365 16665 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.509941 16668 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:29.512912 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:29.515597 17415 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:19:29.516000 17415 pending_rounds.cc:85] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 9
I20260709 13:19:29.516206 17415 pending_rounds.cc:107] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:19:29.516417 17415 raft_consensus.cc:2889] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.519878 16665 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:19:29.523105 21070 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:29.523896 21070 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:29.525591 21070 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.527808 16668 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:19:29.528599 16668 pending_rounds.cc:85] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 7
I20260709 13:19:29.528893 16668 pending_rounds.cc:107] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:19:29.529214 16668 raft_consensus.cc:2889] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:29.530032 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.530622 16668 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:29.531536 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.532583 16669 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:29.534478 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.535035 16663 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.543223 21114 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:29.546118 20953 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:29.547849 20953 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:29.545990 21051 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.552088 21101 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:29.551899 16663 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:29.553260 21103 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:29.553637 21103 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:29.554406 21103 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.557647 20766 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.558583 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:29.559299 16738 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.564040 17412 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:19:29.564427 17412 pending_rounds.cc:85] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 9
I20260709 13:19:29.564642 17412 pending_rounds.cc:107] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:19:29.564873 17412 raft_consensus.cc:2889] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:29.566255 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.564838 16559 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:29.573179 21051 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.574805 21020 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:29.581377 21020 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:29.582397 21020 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.584923 16592 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:29.588821 16738 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:29.597645 16592 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:29.598234 16593 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:29.600253 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:29.605842 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.606599 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.607574 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:29.608134 16669 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:29.608162 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.606592 16666 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:29.609587 16671 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:29.610764 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.613233 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.613785 16665 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:29.575857 21051 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.580650 20889 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.627751 16669 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:29.575002 21114 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:19:29.631909 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:29.636965 20753 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:29.636898 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:29.637486 20753 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:29.637647 16595 raft_consensus.cc:2376] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9. Re-sending same reply.
I20260709 13:19:29.637914 20753 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.633491 21034 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:19:29.640923 21178 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.641489 21178 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.644989 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:29.627242 20938 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.612079 20766 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:19:29.631201 20889 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.658547 20889 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:29.658802 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.014s	sys 0.026s
W20260709 13:19:29.659195 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.014s	sys 0.026s
I20260709 13:19:29.652331 21178 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:29.659456 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 569453us
I20260709 13:19:29.660487 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.661268 20950 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:29.662187 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:29.661098 21178 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:19:29.654987 20938 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:29.654188 16491 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:29.663769 20026 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:29.671469 20026 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:29.671888 20026 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:29.665066 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:29.665482 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:29.653414 21034 consensus_queue.cc:1243] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:29.672866 17409 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:29.674046 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.675079 21118 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:29.681584 16742 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:29.681718 20938 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.682371 20971 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:29.682948 20971 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:29.684031 21051 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:29.684922 20971 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:29.688117 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:29.674075 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:29.718746 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.295135 16588 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:30.296294 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:29.729609 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:29.717886 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.301054 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.302165 16596 raft_consensus.cc:2410] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:19:29.695465 16491 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 5, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 5 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:29.677940 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.633350 20992 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:19:30.348135 20992 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:30.348627 20992 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:29.623327 21019 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:19:29.775358 21103 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:29.724134 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:30.355979 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:30.356369 16593 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.358340 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:29.707484 16590 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:29.719251 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:29.718917 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:29.851432 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:29.890789 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.370993 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.371563 16589 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:29.920791 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:29.928618 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:30.375442 17410 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 9: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:29.961905 17419 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:29.966518 20950 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:30.376559 16595 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:30.377691 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:29.979947 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.378750 21022 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:30.008023 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:30.381675 16595 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:30.019538 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:30.381711 16667 raft_consensus.cc:1217] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Deduplicated request from leader. Original: 11.6->[14.7-16.9]   Dedup: 15.8->[16.9-16.9]
I20260709 13:19:30.079154 21101 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:30.383431 16667 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: [14.7-16.9]
I20260709 13:19:30.383853 21101 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.386335 21101 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:29.682371 20933 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:30.127009 21094 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.387106 20933 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:30.387459 21094 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.388441 20933 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.389518 20933 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:30.389828 21094 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.137109 21117 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:30.145251 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:30.231151 16737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.017s	sys 0.017s
W20260709 13:19:30.391721 16737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.017s	sys 0.017s
I20260709 13:19:30.240551 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:30.392624 16742 raft_consensus.cc:2368] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:30.248063 21034 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:30.395211 21034 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.397598 21034 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:30.254561 20026 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.011s
I20260709 13:19:30.266572 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:30.268726 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:30.270566 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:30.281011 16743 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:30.282444 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:30.292526 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:30.299296 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.315227 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.305881 16491 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 7, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:29.693976 21134 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:30.327836 16663 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:29.725119 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:30.404105 20950 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.407168 16591 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:30.408648 16740 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:30.408202 16491 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:30.409518 16745 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.408253 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:30.411687 16591 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:30.350632 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:30.406591 20950 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:30.379218 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:29.985760 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.350186 20992 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.402930 20026 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.419808 16588 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:30.416862 16587 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:19:30.423362 21101 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:30.369562 21031 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:30.425557 16738 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:30.426766 16596 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:19:30.427312 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.435155 16741 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:30.420730 21094 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:19:30.439160 20996 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.411681 21134 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:30.439757 20996 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.440503 21134 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.442495 20996 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:30.444252 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 93940.2us
I20260709 13:19:30.444720 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:30.447072 17420 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:30.449599 16670 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:30.450048 20992 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:30.426620 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:30.451715 16592 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
W20260709 13:19:30.455793 20753 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.817s	user 0.016s	sys 0.009s
I20260709 13:19:30.465628 17419 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:19:30.422205 16587 pending_rounds.cc:85] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 6
I20260709 13:19:30.465672 16588 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:30.458078 16745 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:30.439010 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:30.468186 17413 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:30.466305 16587 pending_rounds.cc:107] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:19:30.479029 16587 raft_consensus.cc:2889] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.491430 20026 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.467113 21101 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:30.498169 21101 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:30.482587 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62247us
I20260709 13:19:30.508479 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:30.502349 21117 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:30.509416 21117 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.509491 16741 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:30.509910 21117 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:30.516180 20945 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:30.518733 21034 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.519380 20971 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:30.519345 21034 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.520051 20971 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.467726 20753 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.522226 20971 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:30.518594 20943 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:19:30.523146 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96732.5us
I20260709 13:19:30.514753 16561 leader_election.cc:400] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:30.525434 16561 leader_election.cc:403] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.492862 21031 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:30.508659 16594 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:30.540639 16591 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:30.540591 16593 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:30.555577 16590 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:30.561035 16737 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:30.565392 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:30.565668 21114 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:19:30.566339 16666 raft_consensus.cc:2368] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
W20260709 13:19:30.566517 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 208384us
I20260709 13:19:30.551901 21034 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:30.554111 20945 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.557965 20971 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:30.559162 20753 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:30.584106 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:30.586397 16741 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:30.589638 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:30.590314 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:30.593662 16491 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:30.600320 21160 raft_consensus.cc:2764] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:19:30.601089 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:30.601840 16744 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:30.604171 21117 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:30.606951 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64145.2us
I20260709 13:19:30.611948 16559 leader_election.cc:400] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:30.612458 16559 leader_election.cc:403] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:30.612993 21034 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:30.613438 21034 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.613765 21034 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:30.615211 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:30.616189 16662 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:30.618911 21019 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:19:30.622941 21094 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:30.624930 21208 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:30.588686 20945 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.637596 16561 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:30.637044 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:30.647070 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51826.9us
W20260709 13:19:30.651109 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 110290us
I20260709 13:19:30.654909 21031 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:30.657717 21031 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:30.658787 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 101071us
I20260709 13:19:30.659340 20996 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:30.659787 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:30.659973 20996 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.661257 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:30.662186 21035 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:30.662246 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:30.663708 17413 raft_consensus.cc:2410] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:30.664117 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:30.675381 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:30.677331 21022 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:30.680007 20996 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.681106 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:30.682415 21092 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:30.682828 21092 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.682940 21094 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:30.683234 21092 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:30.684224 21041 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:30.685702 20945 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.687290 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 3237) took 1259 ms. Trace:
I20260709 13:19:30.688217 16745 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:19:30.688598 16745 pending_rounds.cc:85] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 8
I20260709 13:19:30.688884 16745 pending_rounds.cc:107] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:19:30.689175 16745 raft_consensus.cc:2889] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:30.690680 20953 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:30.691399 16744 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:30.663105 20943 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:30.693063 20943 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:30.693472 20943 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:30.692353 16491 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 9 to 10, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:30.675082 21019 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:30.687776 16594 rpcz_store.cc:276] 0709 13:19:29.427903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.687861 (+259958us) service_pool.cc:224] Handling call
0709 13:19:30.687055 (+999194us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b54001fb8d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:30.687206 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1846922112}
I20260709 13:19:30.659878 21031 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:30.694833 16491 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 8, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:30.735133 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08001c14a0 after lost signal to thread 15479
W20260709 13:19:30.737073 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08001974a0 after lost signal to thread 16528
W20260709 13:19:30.738957 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.664364 21035 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:30.749372 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.750173 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76152.2us
W20260709 13:19:30.751240 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:30.713982 16491 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 7, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:30.753329 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 79490.4us
I20260709 13:19:30.755170 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:30.713076 20753 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:30.760000 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:30.761385 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:30.769920 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08001c63c0 after lost signal to thread 16620
W20260709 13:19:30.771023 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08003c6320 after lost signal to thread 16621
I20260709 13:19:30.706728 21041 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 13:19:30.773192 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 178077us
I20260709 13:19:30.745518 21035 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:30.742816 21022 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:30.778359 16491 catalog_manager.cc:5697] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:30.779747 21117 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.782096 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 497) took 1103 ms. Trace:
W20260709 13:19:30.797324 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08002a5c80 after lost signal to thread 16695
I20260709 13:19:30.782500 20638 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:30.783279 21099 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:30.785570 21034 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.786648 16666 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
W20260709 13:19:30.805905 16617 debug-util.cc:398] Leaking SignalData structure 0x7b080030f160 after lost signal to thread 16696
I20260709 13:19:30.810401 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:30.818188 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:30.819144 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8740 after lost signal to thread 16769
W20260709 13:19:30.819504 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08003bede0 after lost signal to thread 16770
W20260709 13:19:30.821606 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 158781us
W20260709 13:19:30.824875 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47748.3us
W20260709 13:19:30.821439 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46922.6us
I20260709 13:19:30.810141 20766 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:30.830058 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:30.807001 21099 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:30.833709 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.830884 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55374.4us
W20260709 13:19:30.838596 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08000099e0 after lost signal to thread 17446
W20260709 13:19:30.838960 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08001b1a00 after lost signal to thread 17447
I20260709 13:19:30.840893 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:30.841356 16594 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:30.843700 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:30.844339 20766 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.845046 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:30.847033 21117 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:19:30.848606 21211 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:19:30.827749 20638 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.837841 20945 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:19:30.840240 16634 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:30.859575 20931 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:30.864838 21034 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:30.797713 16665 rpcz_store.cc:276] 0709 13:19:29.667784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:29.668424 (+   640us) service_pool.cc:224] Handling call
0709 13:19:29.668696 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:29.669030 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:19:30.471120 (+802090us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.589026 (+117906us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:30.624042 (+ 35016us) log.cc:844] Serialized 26 byte log entry
0709 13:19:30.624197 (+   155us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:19:30.624444 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 13:19:30.624611 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:30.624706 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:30.768352 (+143646us) raft_consensus.cc:1685] finished
0709 13:19:30.768372 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:30.771058 (+  2686us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":629951,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":73,"raft.queue_time_us":21153,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":26131,"thread_start_us":267356,"threads_started":2,"wal-append.queue_time_us":138471}
I20260709 13:19:30.832898 16491 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 5 to 6, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 6 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:30.870791 16491 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 1 to 7, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:30.838187 21099 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:30.884824 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:30.891312 16617 debug-util.cc:398] Leaking SignalData structure 0x7b08002fe3c0 after lost signal to thread 21125
W20260709 13:19:30.891738 16617 debug-util.cc:398] Leaking SignalData structure 0x7b080040e4c0 after lost signal to thread 21128
W20260709 13:19:30.891950 16617 debug-util.cc:398] Leaking SignalData structure 0x7b080033e3c0 after lost signal to thread 21129
I20260709 13:19:30.822966 21092 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.897698 16669 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:30.843639 21214 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:30.880558 20931 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:30.909041 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:30.902096 21035 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:30.915071 16617 debug-util.cc:398] Leaking SignalData structure 0x7b0800350740 after lost signal to thread 21151
I20260709 13:19:30.916570 20766 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.836796 21022 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:30.922171 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:30.921412 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46627.9us
W20260709 13:19:30.935065 16617 debug-util.cc:398] Leaking SignalData structure 0x7b0800317000 after lost signal to thread 21173
I20260709 13:19:30.939921 17413 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:30.940363 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 93584us
I20260709 13:19:30.933471 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:30.944911 16594 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
W20260709 13:19:30.953249 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:30.954008 21210 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:19:30.969106 21022 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:30.963568 21035 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.971674 16744 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:30.950701 20945 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:30.972273 20945 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:30.971333 20931 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.974040 21092 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:30.972678 20945 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:30.976761 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:30.977275 16742 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:30.977571 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45476.5us
I20260709 13:19:30.986192 21183 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:30.939826 21214 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:30.986632 21183 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:30.994948 21183 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:30.980456 21031 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:30.982290 16491 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:30.998983 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76886.4us
I20260709 13:19:31.008419 21214 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.011442 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:31.015268 21224 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:31.015846 21035 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.015578 16491 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 8, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 8 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:31.023341 21031 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.026136 21031 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:31.022418 20943 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.043792 16742 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:31.048072 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:31.051811 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.055734 16741 raft_consensus.cc:2368] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:31.067112 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:31.075541 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 199595us
I20260709 13:19:31.077653 17413 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:31.077404 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:31.045097 21224 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:19:31.078812 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:31.081530 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:31.084672 21220 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:31.084690 20638 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:31.085292 21220 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:31.085673 21035 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:31.087363 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:31.087715 21227 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:19:31.088274 21226 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:19:31.088668 21031 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:31.089133 21031 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.089474 21031 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:31.089641 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:31.089696 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:31.090524 17409 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:31.052930 21117 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:31.084007 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:31.115455 16668 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:31.115545 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.116209 16739 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:31.086215 21220 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.110255 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:31.112576 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:31.106976 21099 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.120918 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:31.125167 21134 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:31.125474 21160 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.100620 21035 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:31.126675 21035 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:31.127378 21160 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.127727 21035 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.130982 20943 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:31.125752 21134 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:31.135823 21134 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.113196 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:31.149942 21041 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:31.154359 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.155396 16594 raft_consensus.cc:2410] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:31.161124 16667 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:31.162179 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:31.163110 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 247447us
I20260709 13:19:31.176996 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.178285 16634 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:31.182633 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:31.183679 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.184478 16669 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:31.186487 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.187191 16669 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:31.195478 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.196218 17409 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 15: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:31.194834 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.200373 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.200423 16662 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:31.189103 21117 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.207988 21041 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:31.208585 21041 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.209030 21117 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:31.210873 16666 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:19:31.222177 21041 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.226759 16739 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:31.226274 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:31.223162 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.633s (SENT) [suppressed 4 similar messages]
I20260709 13:19:31.239368 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.217886 21224 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:31.247951 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:31.250177 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:31.250427 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:31.250988 16671 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:31.260741 21061 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:31.261358 21061 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.263365 21061 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.260753 21103 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:31.265268 21103 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.267304 21020 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:31.267496 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.267930 21020 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:31.268940 21020 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.251685 21224 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:31.318349 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47042.3us
I20260709 13:19:31.234103 21183 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.249661 21099 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:31.322845 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.357992 16666 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:31.385634 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.386543 16590 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:31.387849 21233 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.112s	user 0.000s	sys 0.004s
I20260709 13:19:31.389700 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:31.393860 20945 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.395161 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:31.395711 21243 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.396185 20945 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:31.397035 16587 raft_consensus.cc:2368] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
W20260709 13:19:31.400328 21242 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:19:31.401587 21234 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:19:31.407320 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.396437 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.387423 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.396364 21243 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.395826 16593 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:19:31.417049 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:31.421293 21240 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:19:31.391949 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:31.422816 16737 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:31.255549 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:31.428121 16662 raft_consensus.cc:606] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.422704 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.428601 16662 raft_consensus.cc:614] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:31.429095 16590 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:31.429847 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:31.431054 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 207917us
I20260709 13:19:31.355330 21019 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:19:31.435899 21224 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.443370 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.454847 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.458978 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.467171 21029 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:31.453279 21208 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.468205 21208 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.469322 21034 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.469885 21034 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.470263 21208 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:31.470635 21103 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:31.492693 16593 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:31.500955 16671 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:31.472821 21243 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:31.519595 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 130017us
I20260709 13:19:31.518162 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:31.524766 16668 raft_consensus.cc:606] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.525219 16668 raft_consensus.cc:614] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:31.473030 21092 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.521806 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:31.486168 21183 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:31.555449 16590 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:31.568881 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 127359us
I20260709 13:19:31.569792 16737 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:31.567270 21216 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:31.565117 20950 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:31.555763 21092 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.578105 21252 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:19:31.584226 21092 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:31.585026 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79104.8us
I20260709 13:19:31.586356 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:31.592749 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:31.593691 21031 consensus_meta.cc:287] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.001s
I20260709 13:19:31.594089 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:31.594794 16667 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:31.577880 20950 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:31.607769 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.609059 16671 raft_consensus.cc:2410] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:19:31.600984 21034 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.620618 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:31.611291 21101 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:31.631208 21101 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:31.632238 21101 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.594107 21031 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.638674 16667 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:31.646636 16561 leader_election.cc:400] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:31.647097 16561 leader_election.cc:403] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:31.662369 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:31.662885 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.663949 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.664819 16666 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:19:31.648433 20996 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:31.678889 21183 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:19:31.679569 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60127.5us
I20260709 13:19:31.662976 21216 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:19:31.681180 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.678342 20950 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.683032 21134 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:31.683706 21031 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.687136 20996 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.691434 21251 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.692070 21251 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.694275 21251 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:31.682654 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:31.689451 20996 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.695250 21134 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:31.695340 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:31.695928 21134 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:31.696117 16662 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.697052 21134 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.698781 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:31.731190 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:31.735456 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.736163 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:31.736444 16742 raft_consensus.cc:2410] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:19:31.736953 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62084.8us
I20260709 13:19:31.703148 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:19:31.713100 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48606.1us
I20260709 13:19:31.722815 21117 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:31.723635 16588 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:31.724473 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:31.744275 16662 raft_consensus.cc:606] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.744633 16662 raft_consensus.cc:614] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:31.740765 16739 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:19:31.761116 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58080.1us
I20260709 13:19:31.772888 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.769109 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.775470 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:31.775795 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:31.776707 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63977.4us
I20260709 13:19:31.779595 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:31.773522 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:31.784047 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.788956 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:31.789538 16593 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:31.789831 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:31.790566 17413 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 15: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:31.792052 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:31.777823 20766 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.799010 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.770152 21029 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.814179 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:31.819629 16708 leader_election.cc:400] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:19:31.793627 21034 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:31.801807 20945 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:31.805016 21134 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:31.844413 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:31.815102 21214 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.814246 20753 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:31.802641 21114 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:19:31.858723 17409 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12.
I20260709 13:19:31.868476 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.805636 21216 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:19:31.877943 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83292.6us
W20260709 13:19:31.878945 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74280us
I20260709 13:19:31.879971 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.884984 21274 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:31.889894 21274 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:31.890339 21274 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:31.884799 21034 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:19:31.903224 21019 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:31.904189 21019 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.904631 16634 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:31.906972 21217 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:31.907483 21217 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.903179 20992 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:31.909241 20992 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.910024 21217 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:31.910732 21094 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:31.911280 21094 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.915020 21236 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:31.915423 21236 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:31.915753 21236 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:31.920478 21230 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:31.877616 21134 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:31.936347 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.938151 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.939043 21041 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:31.939163 21256 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:31.939603 21041 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:31.940106 21041 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:31.942304 21270 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:19:31.944276 20766 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:31.944789 20766 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:31.853590 20945 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:31.946800 20766 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.947656 20945 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:31.953047 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 73162.6us
I20260709 13:19:31.953907 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.971683 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:31.959609 21134 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:31.903540 21020 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:32.029541 21243 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.040563 16745 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:32.041301 16744 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:32.032846 21225 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:32.041903 21243 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:32.045032 20753 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:32.052603 21041 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:32.053628 21019 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:32.042936 21020 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.056974 21020 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:32.064530 17420 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:32.066099 21225 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:32.067088 21224 raft_consensus.cc:993] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:32.067584 21224 raft_consensus.cc:1081] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:32.067365 21225 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.069355 21225 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:32.071700 21094 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:32.074074 20996 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:32.076256 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.081266 21209 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:32.083756 21243 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:32.086206 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 132417us
I20260709 13:19:32.086433 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:32.087545 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:32.088147 16744 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:32.105710 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:32.115813 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:32.116405 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 313204us
I20260709 13:19:32.117426 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:32.119715 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.120361 16671 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:32.120374 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:32.121739 16669 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:32.116667 16744 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:32.127106 16669 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:32.133241 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:32.100600 20992 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:32.137187 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:32.154394 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:32.157349 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:32.158846 16636 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12.
I20260709 13:19:32.159230 16636 leader_election.cc:403] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:32.123356 21256 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.129640 21220 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:32.178145 16667 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:32.185228 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.167280 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:32.133838 21099 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:32.106210 21236 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.191399 16745 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:19:32.204531 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46058.8us
I20260709 13:19:32.209079 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:32.209769 16593 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:32.210091 16593 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:32.211006 21070 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:32.211407 21070 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.211706 21070 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:32.190188 21101 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:32.214167 21041 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:32.214509 21275 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12.
W20260709 13:19:32.213145 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57532.3us
I20260709 13:19:32.221751 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:32.160079 21216 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:32.225077 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.230716 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.231727 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.231972 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.233641 16596 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:32.237306 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:32.243218 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.244508 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.244737 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.245275 16589 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:32.246258 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:32.247125 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:32.248667 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:32.249773 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.245294 16664 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:32.252590 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.194012 21232 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.256433 16594 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:32.256924 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:32.258513 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:32.256275 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:32.260982 16668 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.261417 16668 raft_consensus.cc:614] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:32.228889 16593 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.264377 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.256384 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:32.267683 16595 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:32.272256 16745 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:32.273569 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.275985 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.224538 21101 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.281848 16667 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:19:32.296114 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:32.299633 16587 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:32.300769 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:32.302841 16736 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:32.303032 21277 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:32.304430 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.305559 21101 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:32.309435 21101 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:32.315912 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:32.317166 21278 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:32.331739 16592 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:32.333771 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48679.9us
I20260709 13:19:32.335194 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.277978 21031 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:32.417646 21271 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:19:32.418412 21284 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:19:32.377125 21031 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.402804 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:32.415967 21272 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:19:32.353698 21041 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:32.451259 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:32.435488 21271 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:32.441105 21031 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:32.461964 21271 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:32.467902 21236 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:32.469776 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:32.473372 21274 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.583s	user 0.004s	sys 0.007s
I20260709 13:19:32.473961 21274 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:32.475580 16737 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:32.480458 21117 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:32.487627 16670 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
W20260709 13:19:32.489250 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 214126us
I20260709 13:19:32.501297 16665 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:32.501783 21070 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:32.505995 16593 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:32.506517 21070 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:32.379177 21232 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 13:19:32.510317 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 214590us
I20260709 13:19:32.500674 21291 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:19:32.535104 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.535104 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:32.554438 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 250122us
I20260709 13:19:32.555334 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.558058 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:32.558666 16665 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:32.560120 16665 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:32.541831 20753 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:32.625957 21275 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:32.551162 20950 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:32.759941 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:32.760820 16670 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:19:32.774430 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 219168us
I20260709 13:19:32.775334 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.791234 21298 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:32.783216 21288 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:19:32.809989 21274 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:32.812997 20996 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:32.815827 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:32.816155 17420 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:32.816530 17411 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:32.819065 17411 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:32.823939 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.553877 21134 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.225s
I20260709 13:19:32.778630 16665 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.836584 16665 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:32.814208 20996 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:32.831043 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.847975 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:32.558987 21217 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.264s
I20260709 13:19:32.667817 21278 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.261s
I20260709 13:19:32.833683 21276 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:32.834004 21061 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:32.892496 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 337304us
I20260709 13:19:32.862711 21070 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:32.880643 21216 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:32.822785 21275 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:32.897545 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 343818us
I20260709 13:19:32.905834 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:32.883402 20950 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:19:32.888594 21031 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.891390 21183 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:19:32.896974 21276 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:32.939145 16711 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.023s	sys 0.048s
W20260709 13:19:32.939545 16711 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.023s	sys 0.048s
I20260709 13:19:32.939692 16559 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:32.940642 16667 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:32.941010 21119 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:32.941397 21119 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:32.942214 21119 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:32.943482 21119 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:32.944213 16670 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:32.929464 21216 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:32.999105 21031 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:32.999807 21031 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.000303 21031 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.002352 21031 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:32.971210 20950 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:33.002841 21031 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:19:33.015641 21271 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.000s
I20260709 13:19:33.002372 21099 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:33.018589 21299 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:19:33.035492 21216 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:33.019013 20950 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:33.085907 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.054598 20950 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:33.037568 21271 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.111389 21275 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:33.136423 16670 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:19:33.145886 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:33.148392 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 242654us
I20260709 13:19:33.150154 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 2401) took 1017 ms. Trace:
I20260709 13:19:33.152715 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:33.152829 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:33.167732 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.047205 21099 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.121802 21307 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.180972 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:33.181943 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:33.179309 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:33.185519 16736 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.150645 16662 rpcz_store.cc:276] 0709 13:19:32.132780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.134131 (+  1351us) service_pool.cc:224] Handling call
0709 13:19:32.151488 (+ 17357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:32.251556 (+100068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:32.251583 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:32.278230 (+ 26647us) log.cc:844] Serialized 52 byte log entry
0709 13:19:32.857090 (+578860us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:33.083426 (+226336us) log.cc:844] Serialized 14 byte log entry
0709 13:19:33.149749 (+ 66323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:33.149856 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:33.149911 (+    55us) raft_consensus.cc:1685] finished
0709 13:19:33.149931 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:33.150083 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59967,"raft.queue_time_us":76333,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":75,"thread_start_us":644964,"threads_started":2,"wal-append.queue_time_us":590271}
I20260709 13:19:33.191680 16742 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:19:33.168596 21236 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:33.168881 21305 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.201622 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:33.202491 16739 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:33.215505 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:33.216571 16665 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:33.194710 21034 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.168329 21031 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:33.223843 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:33.224587 16588 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:33.227062 21034 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.229769 21034 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.200240 21020 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.196208 21230 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:33.243103 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90445.8us
I20260709 13:19:33.252362 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:33.252458 21278 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:33.253028 16596 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.255246 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:33.257012 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.258383 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.259315 21034 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.263211 21034 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.265758 21034 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.268765 16588 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:33.261035 16596 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:19:33.270840 16590 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
W20260709 13:19:33.280206 21312 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:19:33.286461 21034 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.287333 21034 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.289908 21034 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.207249 21276 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:33.195294 21307 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.324591 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:33.336618 16665 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:19:33.337492 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 121641us
I20260709 13:19:33.342661 21319 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.347168 21035 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:33.345976 21319 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.347698 21035 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:33.348079 21271 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:33.348626 21035 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.349695 21035 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:33.354333 16665 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:19:33.334057 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.356652 16736 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:33.368933 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:33.216054 21305 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.313769 21020 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.359858 21293 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.358003 21209 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.384740 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:33.367254 21315 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:33.368876 21303 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.403523 16591 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:33.377357 21310 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:33.409777 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48764.6us
I20260709 13:19:33.396593 21209 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.374418 21318 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 13:19:33.464248 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52304.1us
I20260709 13:19:33.481323 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.409782 21303 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.486133 21209 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.467672 21134 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.494015 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:33.406129 21315 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.495147 21134 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.500584 21315 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:33.501155 21286 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:33.501427 21134 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.506580 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.507151 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:33.507233 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.507804 16662 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:19:33.508821 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.509013 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.509637 16665 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:33.510196 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.511077 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.512022 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.507390 16669 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:33.512951 21305 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:33.514412 21325 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:19:33.407081 21293 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.488848 21099 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:33.519695 16589 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:33.524158 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.465032 21310 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.519680 21070 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:19:33.533131 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 166845us
I20260709 13:19:33.534991 16662 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
W20260709 13:19:33.523545 21119 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.003s
I20260709 13:19:33.540587 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.800628 21277 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.588016 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 481) took 1241 ms. Trace:
I20260709 13:19:33.801244 21277 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.801730 16592 rpcz_store.cc:276] 0709 13:19:32.295362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.295816 (+   454us) service_pool.cc:224] Handling call
0709 13:19:33.536275 (+1240459us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b5400320354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:33.536397 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79822080}
I20260709 13:19:33.520262 21305 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.533985 21303 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.652223 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.803846 16663 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:33.658782 21220 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:33.659469 21331 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:33.804864 21331 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.804812 21220 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.807261 21331 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:33.661464 21070 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:33.645233 21297 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:33.808323 21220 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:33.808909 21297 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.810098 21305 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.811426 21297 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:33.607393 21325 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:33.812318 21305 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:33.813743 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64978.4us
I20260709 13:19:33.681367 21336 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:33.684460 21322 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:19:33.814957 21336 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:33.671746 21329 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:19:33.817823 21343 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.692039 21293 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:33.693471 21310 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:33.589246 21319 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:33.699643 21335 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:19:33.633956 21307 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:33.700476 21020 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:33.808418 21070 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.703599 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.705878 21160 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:33.820655 21160 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:33.711913 20950 consensus_meta.cc:287] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.000s
I20260709 13:19:33.718438 21035 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.784904 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.789863 21291 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.818390 21343 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.819489 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:33.704882 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:33.821805 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:33.828586 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:33.833892 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.927032 21291 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:33.828336 20950 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:33.832229 21035 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:33.932359 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79669.7us
I20260709 13:19:33.829918 21298 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.277042 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:34.278209 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:33.832362 21275 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:34.287555 16741 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:19:34.288414 16741 pending_rounds.cc:85] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 7
I20260709 13:19:34.288638 16741 pending_rounds.cc:107] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:19:34.288877 16741 raft_consensus.cc:2889] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:34.285346 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 7.
I20260709 13:19:34.278755 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:34.290627 16593 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:33.829277 21326 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:33.966475 21094 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.279958 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:34.306324 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:33.952380 21341 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:34.281055 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 449199us
I20260709 13:19:34.315754 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.316197 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:34.295481 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:34.297080 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 497109us
I20260709 13:19:34.313391 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.279202 21331 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:34.326490 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.328606 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:34.317915 16634 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:34.331063 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.335709 21331 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.336185 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:34.344183 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.344766 16745 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:34.298013 21061 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.341835 17412 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:33.495429 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:34.324863 21336 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.306185 21298 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.414770 21252 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:34.328928 21275 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.430493 21333 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.323043 21326 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.449636 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.430356 21307 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:34.439582 21277 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:34.427858 21061 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.492473 21061 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:34.493933 21331 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.458123 21310 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.329627 21341 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.518227 17419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 527) took 1694 ms. Trace:
I20260709 13:19:34.334825 21094 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.475869 21296 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:34.511431 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 193613us
I20260709 13:19:34.485107 21293 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:34.524255 21295 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.473127 21319 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:34.525000 21319 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.487336 21160 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:34.527110 21319 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.528746 21275 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.529230 21326 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:34.485890 21274 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:34.530200 21274 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:34.524037 21293 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.534096 21347 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:34.534408 21350 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:34.534629 21347 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.535018 21336 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.535018 21350 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:34.535607 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 3295) took 1317 ms. Trace:
I20260709 13:19:34.535737 21336 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.536216 21350 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.536867 21347 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.537889 21336 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:34.539428 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 2998) took 1360 ms. Trace:
I20260709 13:19:34.539901 16737 rpcz_store.cc:276] 0709 13:19:33.179062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.179172 (+   110us) service_pool.cc:224] Handling call
0709 13:19:33.197940 (+ 18768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.399995 (+202055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.400039 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.400094 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:33.400142 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.539190 (+1139048us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54003edfd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:34.539244 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.539350 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464115584}
I20260709 13:19:34.543387 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.544555 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:34.545764 21326 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:34.550731 21275 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.551280 21275 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.553102 21160 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.553133 21275 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.553421 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 711) took 1261 ms. Trace:
I20260709 13:19:34.553792 21160 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.553835 16738 rpcz_store.cc:276] 0709 13:19:33.291663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.296732 (+  5069us) service_pool.cc:224] Handling call
0709 13:19:33.495673 (+198941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.499746 (+  4073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.499775 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.499819 (+    44us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:33.499850 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.553217 (+1053367us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b54003ee9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:34.553261 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.553366 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8917760}
I20260709 13:19:34.555846 21160 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.518621 17419 rpcz_store.cc:276] 0709 13:19:32.815579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:32.817115 (+  1536us) service_pool.cc:224] Handling call
0709 13:19:32.817387 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:32.817575 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:32.958504 (+140929us) log.cc:844] Serialized 14 byte log entry
0709 13:19:32.959246 (+   742us) log.cc:844] Serialized 14 byte log entry
0709 13:19:34.508551 (+1549305us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b180036c1c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327f79ad70 00007f327f78e293 00007f327f783cea 00007f327f78423a 00007f327fbeea2a 00007f327fbfe0bf 00007f327fbfe038
0709 13:19:34.509882 (+  1331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.509956 (+    74us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:19:34.510021 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.510108 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.510272 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1754,"prepare.queue_time_us":118347,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":25133,"raft.queue_time_us":2862,"spinlock_wait_cycles":12648576,"thread_start_us":140278,"threads_started":1}
I20260709 13:19:34.530679 21310 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.535996 16591 rpcz_store.cc:276] 0709 13:19:33.218150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.218263 (+   113us) service_pool.cc:224] Handling call
0709 13:19:33.219872 (+  1609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.404080 (+184208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.535361 (+1131281us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400052154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:34.535531 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416882944}
I20260709 13:19:34.534386 21271 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:34.477533 21252 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.505971 21307 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.562104 21338 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.483451 21333 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.607497 21278 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:34.707044 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.531181 21274 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.606482 21099 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:34.750101 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:34.717993 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.719712 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.721036 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:34.722935 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 406810us
I20260709 13:19:34.754226 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:34.723891 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 417627us
I20260709 13:19:34.742550 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.755580 17419 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:34.718626 16666 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
W20260709 13:19:34.757936 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 304855us
I20260709 13:19:34.758343 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:34.759697 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:34.761330 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.762250 16666 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
W20260709 13:19:34.762739 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 65008.8us
W20260709 13:19:34.761260 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 67878.8us
W20260709 13:19:34.763674 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:34.765856 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.766112 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:34.766516 16737 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:34.561831 21296 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.619971 21217 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.137s
I20260709 13:19:34.766636 16739 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:34.780351 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:34.700173 21035 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:34.703095 21349 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:34.697996 21070 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:34.762189 21061 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:34.621381 21320 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.142s
I20260709 13:19:34.745802 21297 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:34.816221 21306 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:34.631614 21271 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.755376 21256 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:34.808939 21305 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:34.834882 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 75258.8us
W20260709 13:19:34.835104 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49372.9us
I20260709 13:19:34.839147 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:34.857115 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:34.845332 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:34.763026 21278 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.704336 21338 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.785224 21351 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:34.834757 21346 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:19:34.864394 21298 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:34.872411 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:34.870893 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:34.887140 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 116699us
W20260709 13:19:34.930220 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 407828us
W20260709 13:19:34.933214 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54785.1us
W20260709 13:19:34.941107 17410 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.012s	sys 0.009s
I20260709 13:19:34.793129 21099 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.937451 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
W20260709 13:19:34.941478 17410 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.012s	sys 0.009s
W20260709 13:19:34.942260 16635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.003s	sys 0.018s
W20260709 13:19:34.947459 16635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.003s	sys 0.018s
I20260709 13:19:34.951663 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 3300) took 1609 ms. Trace:
I20260709 13:19:34.953261 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.864442 21331 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:19:34.953797 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.952031 16596 rpcz_store.cc:276] 0709 13:19:33.342035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.342193 (+   158us) service_pool.cc:224] Handling call
0709 13:19:33.342556 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:33.351943 (+  9387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:33.351974 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:33.352012 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:33.352040 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:34.951461 (+1599421us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b5400157a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:34.951506 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:34.951614 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21096192}
I20260709 13:19:34.960196 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:34.954754 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:19:34.960614 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88429us
I20260709 13:19:34.875360 20950 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:34.865677 21305 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:34.876854 21251 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:34.962703 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.964354 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.978489 16667 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
W20260709 13:19:34.966667 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 156829us
I20260709 13:19:34.975543 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.938374 21374 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:34.819445 21349 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.828794 21070 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.857846 21297 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.975837 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:34.950356 21351 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.966426 21350 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:35.017236 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 77084.1us
I20260709 13:19:34.989163 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:34.860937 21306 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.976809 20753 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:34.975845 21298 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:34.983012 21305 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:34.874884 21256 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.997756 21236 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.979374 21353 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:34.975437 21368 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:34.984447 21217 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:19:35.011441 21275 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.001418 21356 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:35.070660 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:35.100173 21353 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:34.979174 21214 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:35.085402 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 717) took 1260 ms. Trace:
W20260709 13:19:35.090014 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 60777.4us
I20260709 13:19:35.010897 21220 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.103879 21353 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.065857 21232 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.105192 21220 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.106235 21232 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.107930 21343 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.108242 21220 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:35.108522 21342 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.378s	user 0.000s	sys 0.001s
I20260709 13:19:35.042079 21350 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.110088 20943 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:35.110523 20943 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.111460 21252 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.112687 21160 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.113319 21306 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.113234 21160 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.115219 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.116943 21350 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.117465 21350 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.117913 21350 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.119743 21160 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.121296 21308 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.121824 21308 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.123853 21308 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.123919 21349 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.062650 21293 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.103282 16740 rpcz_store.cc:276] 0709 13:19:33.824901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:33.825008 (+   107us) service_pool.cc:224] Handling call
0709 13:19:34.280016 (+455008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.432214 (+152198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:34.432249 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:34.432292 (+    43us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:19:34.432331 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.071957 (+639626us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400305f54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:35.072019 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.085321 (+ 13302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349578496}
I20260709 13:19:35.098896 21356 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:35.131542 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 142501us
I20260709 13:19:35.148242 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:35.138394 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 195862us
I20260709 13:19:35.065606 21035 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.145090 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:35.149678 21035 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.150326 21035 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:34.979768 21251 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:35.150642 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.150951 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.151293 17409 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:35.151063 21251 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.152269 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.154119 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.159792 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.093163 21352 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.163744 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.163229 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.158136 21094 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:35.178051 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 147229us
I20260709 13:19:35.091645 21366 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:35.143805 20943 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.108289 21232 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.169731 21360 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.178992 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.182515 21359 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:35.146540 21307 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.067654 21298 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:34.999630 21374 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:35.179868 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81553.3us
I20260709 13:19:35.185674 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:35.171147 21344 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:35.155220 21356 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.183199 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.188427 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:35.187150 21295 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:35.221617 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.222007 21295 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.222469 21295 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.222520 21243 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:35.223133 21243 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.223507 21243 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:35.228377 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.156877 21333 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.234441 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.194365 21271 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.195467 21297 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.195658 21099 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.183375 21308 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:35.197098 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.181012 21366 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.184144 21352 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.249243 21366 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:35.249389 21382 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.250020 21382 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.250718 21352 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.179761 21349 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.197660 21360 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.252218 21382 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.253159 21360 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:35.253384 21391 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:19:35.198580 21359 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.198560 21357 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.253813 21230 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:35.254251 21230 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:35.254453 21362 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:19:35.254124 21357 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.255369 21359 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.255138 21230 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.256373 21357 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:35.208580 21376 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:19:35.209290 21220 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.257412 21220 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.259547 21220 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.209753 21364 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.260237 21364 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.211041 21278 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.211943 21307 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:35.263214 21338 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:35.212131 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49243.9us
I20260709 13:19:35.263593 21291 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.264174 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.264530 21070 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.264700 21232 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.265384 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.265265 21232 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.266265 21298 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.267256 21307 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:35.218118 21350 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.126466 21368 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.126854 21236 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.268051 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.268575 21307 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:35.269335 21236 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.269351 21307 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:35.270453 21307 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:35.269371 21368 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.271304 21310 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.192281 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.235100 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:35.272300 17412 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.242537 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.271224 21374 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.156198 21275 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.277046 21275 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.246626 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.248301 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.248739 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.232538 21326 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:35.278761 21351 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.188010 21296 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.279029 17413 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:35.217190 21214 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:35.284024 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.280843 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.288147 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.288820 17413 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:35.287899 21303 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.267496 21363 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:35.290155 21303 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.256394 21230 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:35.291448 21232 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.292285 21303 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.292639 21305 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.293529 21101 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.294028 21101 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.289826 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:35.316845 16744 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:35.305298 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 113076us
W20260709 13:19:35.320264 21381 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.092s	user 0.002s	sys 0.000s
I20260709 13:19:35.304220 21363 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:35.323822 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.324978 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.326839 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.325675 21349 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.328047 17413 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:35.328011 21349 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.329147 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:35.325735 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.330581 21349 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.334724 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.340113 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.305262 16559 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.345786 21256 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.347100 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.347357 21341 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.347795 16737 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:35.348016 21364 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.349332 21390 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.303836 21243 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.294960 21388 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.356101 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.356302 21243 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:35.356643 21370 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.146s	user 0.003s	sys 0.000s
I20260709 13:19:35.356817 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:19:35.361694 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.363216 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 529) took 1387 ms. Trace:
I20260709 13:19:35.365609 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.244072 21251 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.349861 21390 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.368355 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:35.369285 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46293us
I20260709 13:19:35.369678 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.370502 17412 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:35.370692 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:35.373605 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55506.3us
I20260709 13:19:35.375160 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.356420 21388 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.379824 21388 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.267272 21356 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.411224 21356 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.296110 21101 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.381318 21251 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.294750 21305 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.301494 21308 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:35.414388 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.368251 21326 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.415796 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.384310 21390 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.384836 21353 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.386442 21394 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.387977 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.299718 21307 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.389178 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.425854 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.426376 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:35.426910 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 57059.8us
I20260709 13:19:35.389896 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.372910 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1568) took 1081 ms. Trace:
I20260709 13:19:35.296438 21310 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.399608 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.400028 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.402913 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.404350 20950 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.404840 21252 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:35.404781 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.407960 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.409350 20945 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.302345 21292 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:35.288339 20943 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.440248 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:35.423300 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 118101us
I20260709 13:19:35.441184 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.331502 21363 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.425989 16593 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:35.432333 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.441227 16588 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:35.443387 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.436116 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.439546 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.400597 21214 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:35.429100 21305 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:35.455649 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.430034 21390 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:35.417465 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.440387 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.388550 21366 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.442148 16744 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:35.442770 16590 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:35.428297 21357 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.428023 16741 rpcz_store.cc:276] 0709 13:19:34.291315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.291545 (+   230us) service_pool.cc:224] Handling call
0709 13:19:34.291862 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:34.965816 (+673954us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:19:34.979275 (+ 13459us) log.cc:844] Serialized 14 byte log entry
0709 13:19:35.232668 (+253393us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:19:35.232721 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:19:35.232763 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.319483 (+ 86720us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:35.372704 (+ 53221us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400426154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:35.372754 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.372859 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29133,"prepare.queue_time_us":13300,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":39,"raft.queue_time_us":78,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":5753,"spinlock_wait_cycles":1670362496,"thread_start_us":252504,"threads_started":1,"wal-append.queue_time_us":248205}
I20260709 13:19:35.440505 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.464649 16589 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:35.440119 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.464892 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.451159 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.451642 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.466764 16664 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:35.400270 21035 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.455091 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:35.483747 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.416119 21326 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.428640 21350 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:35.440331 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.489542 16667 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.456741 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.449216 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:35.469627 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52252.1us
I20260709 13:19:35.455457 21295 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.490608 16666 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:35.468711 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.492234 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.471246 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.491396 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.457659 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:35.476199 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.453106 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.494114 16587 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:35.495275 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.495996 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.496309 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.497174 21271 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:35.497973 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:35.498518 16596 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.499037 16671 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:35.443914 20950 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.447093 21232 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:35.493436 16663 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:35.494378 16634 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.513000 17420 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:35.508448 16596 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:35.510349 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.510613 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:35.509596 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 55535.6us
I20260709 13:19:35.515812 16666 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:35.511778 16667 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:35.443289 21236 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.514832 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.517021 16663 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:35.521345 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.523748 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.456066 21305 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.516279 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.526306 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.455981 21292 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.524734 16591 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:35.467101 21356 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.532521 16666 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:19:35.535202 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.539005 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:35.536464 16668 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:35.537017 17415 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:35.532640 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.471136 21357 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.446682 21353 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.457446 21252 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.463593 20945 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.595958 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.503877 21274 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:35.576941 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:35.503748 20950 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.618283 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.617717 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.619815 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.595997 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.549537 16744 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:35.619774 16663 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:35.637523 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.590147 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:35.637750 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 98822.8us
I20260709 13:19:35.659886 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:35.658972 16666 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:35.544900 21292 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.538616 21305 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:35.663825 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 87009.4us
I20260709 13:19:35.592801 21035 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.665839 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.666543 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.667254 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.538420 21225 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.674382 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.441672 21394 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.546290 21217 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:35.676931 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.504752 21350 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.671219 21360 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:35.727257 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:35.728011 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.728788 17420 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:35.728899 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.680331 16667 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:35.730134 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.682183 16671 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:35.544971 21368 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.591476 20943 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:35.681337 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.731525 16741 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:35.590451 21385 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.685205 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:35.735910 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.618237 21274 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.732757 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.734793 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.743847 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.737952 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.731704 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.528476 21236 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.746141 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:35.739639 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 54508.9us
I20260709 13:19:35.638586 21363 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:35.741572 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64711.4us
I20260709 13:19:35.744267 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.681922 21225 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.761185 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.761471 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.761807 16745 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:19:35.762079 16666 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:35.762982 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:35.764096 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:35.680621 21374 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:35.739776 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:35.752835 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.755362 16559 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:35.679221 21295 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.765285 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:35.744056 21274 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.771682 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.773713 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:35.785789 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:35.773860 16634 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.765954 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:35.779280 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:35.787072 16671 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:35.744769 21232 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:35.791041 21364 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:35.791183 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.791409 21101 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:35.792021 21101 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:35.792658 16738 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:19:35.793038 16738 pending_rounds.cc:85] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 7
I20260709 13:19:35.793293 16738 pending_rounds.cc:107] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:19:35.793635 16738 raft_consensus.cc:2889] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:35.793123 21101 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.787621 21094 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:35.797853 21331 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:35.798642 16743 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:35.797411 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:35.800101 21331 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.800475 21331 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.800902 21331 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:35.672731 21356 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.805629 21394 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.806499 21356 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.807216 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:35.808034 21349 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:35.808429 21349 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.808817 21349 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:35.811781 21252 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.813406 21326 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:35.813957 21316 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:35.743387 21310 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.814756 21310 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.815209 21310 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:35.765230 21225 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:35.764986 20943 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:19:35.823644 20943 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:35.824048 20943 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:35.824175 21353 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.824952 20943 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:35.826103 21236 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.826216 20943 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.771243 21297 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:35.831238 21297 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.831648 21297 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.834008 21331 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.731189 21368 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.788546 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:35.789932 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.782619 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:35.839519 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56972.5us
I20260709 13:19:35.781363 16666 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:35.836647 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.801870 21307 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:35.844496 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:35.845221 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56749.2us
I20260709 13:19:35.845659 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:35.842460 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:35.841262 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.838423 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:35.777457 21392 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:35.853173 21295 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:35.853593 21295 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.853965 21295 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:35.759783 21385 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.859555 21385 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:35.860749 21395 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:19:35.777079 21374 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:35.839373 16741 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:35.849926 16634 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:35.865581 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:35.883746 16742 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:35.880681 20950 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.890923 16667 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:35.892079 16588 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:35.892673 21360 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:35.893252 21360 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:35.895004 21364 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.895433 21364 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:35.895593 21400 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.095s	user 0.004s	sys 0.000s
I20260709 13:19:35.895885 21364 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:35.896776 21368 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.898411 21350 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.862254 21383 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:35.900524 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:35.862501 21243 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.869040 21236 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:35.894210 21360 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.916113 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.907716 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:35.903455 21397 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 13:19:35.886137 20950 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.888348 21374 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.925684 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.926136 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.927977 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:35.913251 21243 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.933720 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.943507 16593 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:35.933938 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.940385 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:35.944375 16587 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:35.921852 21236 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:35.965675 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:35.940881 21383 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:35.936129 20950 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:35.363593 17410 rpcz_store.cc:276] 0709 13:19:34.278822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:34.286991 (+  8169us) service_pool.cc:224] Handling call
0709 13:19:34.324156 (+ 37165us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:19:34.408500 (+ 84344us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:19:34.959104 (+550604us) log.cc:844] Serialized 14 byte log entry
0709 13:19:35.247399 (+288295us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:19:35.253642 (+  6243us) log.cc:844] Serialized 52 byte log entry
0709 13:19:35.253789 (+   147us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:19:35.281873 (+ 28084us) log.cc:844] Serialized 14 byte log entry
0709 13:19:35.283628 (+  1755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:35.283694 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:35.362915 (+ 79221us) raft_consensus.cc:1685] finished
0709 13:19:35.362935 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:35.363159 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84297,"prepare.queue_time_us":761846,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":71,"raft.queue_time_us":172543,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":3144,"thread_start_us":768156,"threads_started":3,"wal-append.queue_time_us":376957}
I20260709 13:19:35.941211 21366 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.920077 21368 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:19:35.946182 21357 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:35.963855 21236 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:35.937474 21349 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:35.991595 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:35.997241 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:35.989553 21295 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.001963 21295 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:35.975514 21383 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:36.005038 16744 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:35.964043 21392 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:19:35.948508 21243 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:36.009933 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54769.7us
I20260709 13:19:36.013779 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:36.015663 21251 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:36.016188 21251 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:36.016685 21251 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:36.017313 16491 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 220) took 29749 ms (client timeout 15000 ms). Trace:
I20260709 13:19:36.027539 21251 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.037185 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:36.040491 21214 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:36.041100 21251 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.041045 21214 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:35.998723 20945 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.049191 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:36.041714 21214 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.006644 21383 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:36.011583 21295 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.068637 21295 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.005792 21297 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.073091 21297 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.095862 21295 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.100014 21366 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.111454 21366 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.113554 21366 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.142096 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.042510 21310 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.152793 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.067529 21368 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:19:36.179430 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:36.180130 16667 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:36.153774 21388 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:36.200313 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:36.224582 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:36.225785 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44862.8us
I20260709 13:19:36.206143 17415 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:36.189169 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.229167 16666 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:36.186094 16665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.086s	sys 0.091s
W20260709 13:19:36.230067 16665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.089s	sys 0.102s
I20260709 13:19:36.163478 21331 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.201740 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:36.255318 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53657.4us
I20260709 13:19:36.257869 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.265650 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.266294 16592 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:36.266430 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.267362 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.267969 16591 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:36.260675 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.270886 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.235147 21357 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:36.180824 21392 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:36.275594 16593 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:36.287940 16488 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.292235 16488 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.292467 16591 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:36.295010 16488 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.335629 16488 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 3 to 9, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.337335 16488 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 9, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 9 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.259398 21316 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:36.268520 21356 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:36.340742 21316 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.341284 21316 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.344440 21349 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.349615 20950 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.278506 21333 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:36.353004 21236 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:36.356662 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.358497 16588 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:36.361363 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:36.310519 21305 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.003s
I20260709 13:19:36.280138 21035 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:36.262777 21225 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.312743 21271 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:36.315738 21292 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.000s
I20260709 13:19:36.324565 21310 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:36.371709 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.372278 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.180594 21364 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:36.017874 16491 rpcz_store.cc:269] 0709 13:19:06.267454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:06.267758 (+   304us) service_pool.cc:224] Handling call
0709 13:19:06.805296 (+537538us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:19:06.965549 (+160253us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfe9c 00007f327cac00f6 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:19:36.017166 (+29051617us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:31.388871 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:31.389602 (+   731us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:31.389658 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:19:31.389693 (+    35us) tablet.cc:662] Decoding operations
0709 13:19:31.390910 (+  1217us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:31.391001 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:31.391027 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:19:31.394886 (+  3859us) tablet.cc:701] Row locks acquired
0709 13:19:31.394932 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:19:31.395101 (+   169us) write_op.cc:270] Start()
0709 13:19:31.395251 (+   150us) write_op.cc:276] Timestamp: P: 1783603171395041 usec, L: 0
0709 13:19:31.395285 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:19:31.491447 (+ 96162us) log.cc:844] Serialized 5096 byte log entry
0709 13:19:31.685858 (+194411us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbdfe28 00007f328361a581 00007f3283619d58 00007f328361e9f1 00007f328361e9a9 00007f328361e939 00007f328361e901 00007f328361db2d 000000000038f354 000000000038f199
0709 13:19:32.181649 (+495791us) op_driver.cc:464] REPLICATION: finished
0709 13:19:32.490105 (+308456us) write_op.cc:301] APPLY: starting
0709 13:19:32.490220 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:32.782709 (+292489us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:32.782755 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:32.937721 (+154966us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:32.938165 (+   444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:19:32.938252 (+    87us) write_op.cc:312] APPLY: finished
0709 13:19:32.938662 (+   410us) log.cc:844] Serialized 121 byte log entry
0709 13:19:32.948685 (+ 10023us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:32.948859 (+   174us) write_op.cc:454] Released schema lock
0709 13:19:32.948939 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:32.949424 (+   485us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:32.949456 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":266730,"reported_tablets":51,"spinlock_wait_cycles":126514048,"child_traces":[["op",{"apply.queue_time_us":308011,"apply.run_cpu_time_us":24911,"apply.run_wall_time_us":459323,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":240290,"prepare.run_cpu_time_us":15742,"prepare.run_wall_time_us":321699,"raft.queue_time_us":390315,"raft.run_cpu_time_us":26735,"raft.run_wall_time_us":173597,"replication_time_us":783665,"spinlock_wait_cycles":58083456,"thread_start_us":708680,"threads_started":4,"wal-append.queue_time_us":294234,"wal-append.run_cpu_time_us":69413,"wal-append.run_wall_time_us":2486129}]]}
I20260709 13:19:36.309501 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:36.375502 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 66098.2us
I20260709 13:19:36.377408 16669 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:36.361439 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.361363 16667 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:36.366781 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.379887 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.321902 21243 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.370508 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.365453 21374 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:36.327828 21274 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.000s
I20260709 13:19:36.367964 21305 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.382754 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.412796 17414 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:36.382891 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.367125 16666 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:36.388919 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.404395 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:36.415400 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.416712 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:36.416900 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.367867 21035 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:36.417497 21350 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:36.417843 21275 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:36.418646 21350 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:36.418330 21035 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:36.385116 21307 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.528s	user 0.004s	sys 0.006s
I20260709 13:19:36.414125 16588 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:36.403601 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.424643 16593 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:36.419086 21350 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:36.430225 16594 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:36.410636 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.343941 16488 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 11, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 11 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.374544 21236 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.381876 21295 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:36.429395 16588 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:36.431442 16595 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:36.433947 16592 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:36.436046 21214 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.440047 16488 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.441193 21035 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:36.445318 20950 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.459594 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.442265 21316 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.460783 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.468091 16737 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:36.468222 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.418357 21275 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:36.394209 21366 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:19:36.374779 21292 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.482764 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:36.477478 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.486558 16591 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:19:36.497150 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 226107us
I20260709 13:19:36.498073 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:36.499329 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.420804 21274 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.516049 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:36.477043 21271 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:36.511286 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.513103 21383 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.522918 16740 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:36.535195 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.536641 21243 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.539350 21383 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.544843 21214 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:36.547003 21374 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:36.547451 21308 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:36.548377 21225 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:36.548826 21393 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:36.549239 21393 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:36.550153 21275 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.552204 21292 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:36.560238 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 56833.1us
I20260709 13:19:36.552886 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:36.560927 17414 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.553453 16739 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:36.556124 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53267.6us
I20260709 13:19:36.553306 21274 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:36.564565 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46043.2us
W20260709 13:19:36.555333 21406 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:19:36.564872 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 66912.1us
I20260709 13:19:36.560984 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.575364 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:19:36.576217 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.576454 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.568744 16663 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:36.568943 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.565804 16593 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:36.578073 16594 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:36.578505 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 200762us
I20260709 13:19:36.580109 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.580550 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.582407 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.584470 16666 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:36.585330 16668 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:36.588235 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.588830 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:36.589022 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.555294 21230 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:19:36.563316 21214 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:36.597637 16592 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:36.603020 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.605297 21316 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.608757 21217 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:36.595479 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:36.615222 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 113423us
I20260709 13:19:36.616431 21390 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:36.616519 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.617252 21412 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:36.617378 16596 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:36.629905 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 1587) took 1249 ms. Trace:
I20260709 13:19:36.642647 16737 rpcz_store.cc:276] 0709 13:19:35.363468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.363627 (+   159us) service_pool.cc:224] Handling call
0709 13:19:36.612963 (+1249336us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002b98d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:36.613078 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2392138496}
I20260709 13:19:36.572898 21251 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:36.645095 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:36.641047 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44908.4us
I20260709 13:19:36.644912 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:36.634982 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.645382 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:36.591853 21216 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:36.592236 21307 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:36.643955 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.667279 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:36.653995 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 3028) took 1218 ms. Trace:
I20260709 13:19:36.671080 16740 rpcz_store.cc:276] 0709 13:19:35.435909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.440950 (+  5041us) service_pool.cc:224] Handling call
0709 13:19:36.653789 (+1212839us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400306954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:36.653927 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477535488}
I20260709 13:19:36.654098 16743 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:36.657518 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.660382 21366 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:36.664309 16662 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 1903) took 1214 ms. Trace:
I20260709 13:19:36.686889 16662 rpcz_store.cc:276] 0709 13:19:35.450245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.450364 (+   119us) service_pool.cc:224] Handling call
0709 13:19:35.450658 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:35.771722 (+321064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:35.771751 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:36.042736 (+270985us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b2c00010da8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:19:36.055983 (+ 13247us) log.cc:844] Serialized 26 byte log entry
0709 13:19:36.368062 (+312079us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:19:36.368114 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.438342 (+ 70228us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540001a754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:36.438394 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:36.664120 (+225726us) raft_consensus.cc:1685] finished
0709 13:19:36.664140 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:36.664256 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11146,"raft.queue_time_us":11255,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":10777,"spinlock_wait_cycles":714213376,"thread_start_us":288831,"threads_started":1,"wal-append.queue_time_us":342905}
I20260709 13:19:36.550055 21393 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:36.644348 21403 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:19:36.639086 21225 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:36.557993 21350 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.642486 20950 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:36.614081 21368 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.652060 21251 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.664191 21316 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:36.658648 21412 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:36.657817 21216 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:36.752800 21251 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:36.754053 17414 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:36.766657 21368 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.767140 21271 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:36.694382 21236 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:19:36.768162 16668 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:36.655249 21292 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:19:36.769820 21364 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.773581 21275 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:36.549265 16488 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 8, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 8 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.774489 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:36.781960 21316 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:36.786842 16488 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.787983 21316 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.788873 16488 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 8, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 8 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.652634 21214 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:36.777392 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 132562us
W20260709 13:19:36.741436 21399 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:19:36.693708 21366 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.779805 21349 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.781414 21271 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:36.777395 21305 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.663904 21411 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.124s
I20260709 13:19:36.675971 21094 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:36.689260 21230 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:19:36.689909 21353 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:36.778913 21216 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.794068 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.794947 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.794677 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.795344 21349 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.802053 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1592) took 1051 ms. Trace:
I20260709 13:19:36.802394 21214 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.814590 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:36.817809 21333 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:36.825539 21333 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:36.823092 16743 rpcz_store.cc:276] 0709 13:19:35.750664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:35.800376 (+ 49712us) service_pool.cc:224] Handling call
0709 13:19:36.653918 (+853542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:36.672142 (+ 18224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:36.801866 (+129724us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400306950. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:19:36.801997 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962429824}
I20260709 13:19:36.826498 21333 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.827917 21333 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:36.828552 21349 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.830299 21341 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:36.830739 21341 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:36.819237 21412 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.813134 21353 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.834164 21353 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.835994 21350 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:36.831713 21341 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.840049 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:36.859261 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 192513us
W20260709 13:19:36.867564 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 219515us
I20260709 13:19:36.868848 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:36.874178 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 222459us
I20260709 13:19:36.874586 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.835523 21251 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.886051 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:36.881378 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.886821 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:36.887945 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:36.892377 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:36.893167 16737 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:36.889011 16668 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:36.899318 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:36.876327 21432 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:36.888002 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.915138 21432 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:36.915661 21432 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:36.923254 16488 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 11, leader changed from <none> to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 11 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.924222 21216 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.925724 21368 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:36.926594 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51475.1us
I20260709 13:19:36.928156 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:36.928845 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:36.932842 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:36.933130 21374 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:36.925483 16488 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 2 to 8, term changed from 9 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to <none>, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 12 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.936239 21366 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:36.936134 16488 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 9, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:36.939663 21432 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.875195 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.940214 16741 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.943323 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.940030 16596 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:36.940973 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:36.942667 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:36.945533 16668 raft_consensus.cc:606] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:36.945902 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:36.951995 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76877.4us
I20260709 13:19:36.937898 16488 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 11, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 11 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:36.956480 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:36.963569 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:36.960062 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.965818 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.947264 21243 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:36.976363 21390 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:19:36.977391 20943 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:36.977974 21394 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:36.978576 21394 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.978600 20943 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:36.979010 21360 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:36.979507 20943 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:36.979686 21360 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:36.980350 21434 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.112s	user 0.002s	sys 0.000s
I20260709 13:19:36.980585 20943 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.981887 21360 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.981974 20943 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:36.983155 21393 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:36.983719 21326 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:36.983860 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.985037 21326 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:36.985100 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:36.985941 21326 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:36.977667 21256 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:36.990130 21293 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:36.994441 16667 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:36.994916 21274 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:36.995306 21274 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:36.995817 16589 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:36.996238 21274 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:36.997517 21274 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:36.977643 21390 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:36.978096 21363 raft_consensus.cc:2764] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:19:37.004966 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.003052 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:36.955930 21217 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:36.999876 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.980996 21394 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.000530 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.954218 21295 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.027972 16590 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:37.028584 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.028825 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.029625 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.031016 21341 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:36.999538 21243 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:37.032732 21243 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.034684 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:37.035324 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.006758 21271 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.036268 16744 raft_consensus.cc:2410] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:37.001987 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.037436 21271 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:37.037755 16591 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 10: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:36.999577 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.957186 21275 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:19:37.003237 21390 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.039127 17411 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:37.039433 21275 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:37.039906 21275 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:37.039758 21390 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.040740 21275 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:36.998413 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:36.986555 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.041119 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:37.042176 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55687.3us
I20260709 13:19:37.042281 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:36.951640 21316 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:37.043144 21316 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:37.044611 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.045213 21316 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:37.008175 17415 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:37.045526 21411 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:37.008440 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.048130 17414 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.008927 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:36.977417 21432 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.944312 21368 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:19:37.016301 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.017231 16737 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:37.018083 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:37.062846 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.018595 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.063882 16592 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:37.018748 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.019780 16740 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.020346 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.021389 16742 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:37.022451 21216 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:36.967357 21252 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:19:36.964465 21413 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:19:37.025554 21251 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:36.990659 21293 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.041075 21390 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.070392 21293 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.051328 16594 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:37.053161 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.075624 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.054790 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.081004 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.081766 16744 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.049966 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.084084 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.085687 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.087073 21357 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.087982 21357 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.088732 21357 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:37.089154 21357 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.089546 21357 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.071170 16587 pending_rounds.cc:85] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 9
I20260709 13:19:37.091596 16587 pending_rounds.cc:107] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:19:37.091938 16587 raft_consensus.cc:2889] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:37.072247 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.094707 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.095814 21232 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.078373 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.102485 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.103626 21251 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.065796 16743 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 4.
W20260709 13:19:37.110172 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 93971.8us
I20260709 13:19:37.112285 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:37.110375 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 61971.5us
I20260709 13:19:37.113370 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:37.114672 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 95234.4us
I20260709 13:19:37.106739 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.117834 16744 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:37.121330 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.133641 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.133911 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.145754 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.170629 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.175303 16739 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:19:37.129454 21292 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:37.066314 21383 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:19:37.123047 21368 consensus_queue.cc:1243] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:19:37.134435 21251 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.144026 21293 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:37.177232 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56016.7us
I20260709 13:19:37.135296 21275 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.194814 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.134709 21430 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:37.198312 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:37.190250 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76942.4us
I20260709 13:19:37.150319 21316 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:37.204393 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.203599 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.209894 17414 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:37.199524 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.215071 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:37.215847 16665 raft_consensus.cc:606] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.216264 16665 raft_consensus.cc:614] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:37.216593 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.170495 21390 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:19:37.192355 21366 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:37.221329 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.220561 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.201373 21368 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:37.229388 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.204258 21349 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.224501 21331 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:37.179582 21101 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:19:37.237844 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:37.238418 16665 raft_consensus.cc:606] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.234697 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:37.233079 21316 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:19:37.206749 21293 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.231884 21383 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.237017 21394 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:37.242023 21430 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:37.239086 21331 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:37.273676 16737 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:37.264190 21368 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:37.274420 21331 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:37.243623 21353 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.267249 21394 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:37.290979 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.296859 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:37.307883 21439 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:19:37.308444 21293 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:37.283988 21368 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:37.309438 21368 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.290333 21214 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:19:37.186113 16488 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 9, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 9 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:37.316363 16669 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:37.317992 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.327392 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:37.328660 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.328925 16666 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
W20260709 13:19:37.332886 21436 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:19:37.340345 16737 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:37.341543 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.342296 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.317791 21230 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:19:37.293479 21292 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:37.319576 21394 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.347601 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:37.349334 21442 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:19:37.358407 21243 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.363016 17415 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.356423 21349 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.341764 21035 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.365916 21232 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.366240 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:37.369053 21374 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:37.369491 21374 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.377434 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.328871 21353 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:37.361687 21357 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.379493 16742 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:37.384478 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.387208 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:37.433804 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:37.469197 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.929s (SENT) [suppressed 1 similar messages]
I20260709 13:19:37.374229 21295 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:19:37.473692 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 89295.2us
I20260709 13:19:37.475729 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.476797 16667 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:37.478348 21392 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:37.479180 21392 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.481385 21392 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:37.473145 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 86023.5us
I20260709 13:19:37.379077 21349 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.483397 21349 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:37.486228 16662 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:37.370330 21374 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.461480 21230 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:37.514309 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.399221 21271 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:37.472839 21341 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:37.472653 21099 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:37.524204 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.505573 21310 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.503491 21214 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:37.528460 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.474942 21295 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:37.487758 21363 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:37.529327 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.468966 21035 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:19:37.530241 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.531867 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.532899 21099 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.577491 21385 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:37.543452 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.577984 21385 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:37.532738 21341 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.544466 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.582252 21341 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:37.583912 16665 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.544857 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.585203 16740 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:37.589030 16740 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:37.589836 16740 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.590951 16740 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.586648 16736 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:19:37.586417 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.587649 16663 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:37.583678 16596 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:37.598955 16589 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:37.618360 21243 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:37.623555 21350 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:37.627635 21350 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.628926 21357 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:37.629761 21214 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.622788 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.631524 21357 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:37.632082 21214 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.633353 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.633791 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.638691 21357 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.638913 21368 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.641410 21368 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:37.583329 21099 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:37.606856 16740 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:37.610076 20945 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:37.611231 20638 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:19:37.614909 21411 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
I20260709 13:19:37.629761 21350 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.603914 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.489125 16488 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 12 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:37.642350 16670 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:37.645797 16596 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:37.727977 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.646260 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.646667 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.648458 21433 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:37.649497 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.650499 16590 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:37.579005 21385 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.655073 21390 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:37.658890 16671 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:37.659355 21331 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.658607 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.660053 21428 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:37.653734 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.669044 21374 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:19:37.602425 21435 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.433s	user 0.000s	sys 0.002s
W20260709 13:19:37.671095 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 55497.3us
I20260709 13:19:37.671280 16736 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:37.671166 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.757382 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.758062 21252 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:37.758677 21390 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:37.529479 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:37.759141 21099 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:37.551852 21440 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.139s
I20260709 13:19:37.761688 21429 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:37.762003 21440 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:37.762440 21440 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.538549 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.763911 16587 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:37.558729 21448 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:37.674358 21295 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.765933 21440 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.765316 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.692271 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:37.693408 16594 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:37.772236 21412 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:37.772929 21412 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.775023 20950 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:37.775187 21409 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:37.775839 21409 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:37.701805 16738 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:37.709885 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.522962 21271 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.739344 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.741370 16739 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:37.707417 21305 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:19:37.779014 21271 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.695273 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.707355 20945 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:37.780118 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84948us
I20260709 13:19:37.780529 20945 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.713752 16744 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:37.707566 20638 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:37.756742 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:37.782312 20638 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.783512 20638 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.521040 21432 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.784183 21432 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:37.739272 21411 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:37.762465 16745 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:37.787779 16668 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.787871 21331 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.789554 20945 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:37.791766 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:37.792654 21101 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:37.793213 21101 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.793351 21385 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.766379 21295 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:37.796732 21454 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:19:37.787460 21411 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.683259 17414 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:37.749792 21392 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:19:37.788172 16668 raft_consensus.cc:614] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:37.801054 21409 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.802165 20943 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:37.802603 20943 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.802786 21409 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:37.803421 20943 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.742655 21394 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.804553 20943 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.738682 21236 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:19:37.745615 16589 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:37.806124 21394 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.806825 16671 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:37.811036 21411 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:37.811770 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:37.813932 21429 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:37.814551 21429 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:37.815176 21440 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:37.815589 21429 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.817422 21429 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.811854 21411 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:37.818113 21411 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:37.819214 21412 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.804821 16745 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:37.819499 21292 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.819996 21292 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.780905 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:37.821012 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 351823us
W20260709 13:19:37.821086 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64432.1us
I20260709 13:19:37.794229 16592 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:37.823987 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.828809 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.830112 16666 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:37.830569 16669 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:19:37.830703 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.831094 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.831753 16667 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:37.832425 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:37.832937 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 67756.8us
I20260709 13:19:37.833621 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:37.835526 16662 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:37.838408 21236 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:37.839118 21035 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:37.834973 21217 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.824642 21363 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:19:37.842756 21305 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:37.843102 21428 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:19:37.843417 21363 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:37.843693 21428 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:37.843880 21363 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.844259 21428 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:37.844748 21363 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.846002 21363 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.835395 21433 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:37.846944 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.847321 21433 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:37.847740 21433 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.848119 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.849241 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.850060 21419 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.850685 21419 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:37.851263 21419 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:37.851843 21419 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:37.759653 21099 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.857394 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:37.860834 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:37.794269 21101 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.862938 21243 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.863334 21101 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:37.825114 21412 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:37.869136 16671 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:37.868316 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.861145 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.834216 21411 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.896989 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.897325 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.898362 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.911017 21225 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:37.853021 21099 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.877039 21305 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:37.912063 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 43822.5us
I20260709 13:19:37.912623 21099 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:37.913394 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.913566 21368 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:37.913414 21305 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:37.915748 21271 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:37.916209 21383 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.916045 21305 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.917438 21243 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:37.911523 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:37.920310 21331 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.923353 21411 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:37.924079 21350 consensus_queue.cc:1175] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "efa00eb5e7404570958fdebb6cd077e8" responder_term: 12 status { last_received { term: 9 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: [11.8-11.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:37.858466 21217 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:19:37.926705 17420 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:37.931941 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.933984 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.937004 17412 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.940025 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.941310 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:37.928812 21035 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:37.943828 16590 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:19:37.943619 21035 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:37.945892 21035 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.947894 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:37.948540 21292 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:37.949301 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.949731 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.949954 17420 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:37.950496 17412 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:37.951771 17410 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:37.953109 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:37.954531 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:37.956118 17411 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:37.957134 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.958940 21035 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:37.959169 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:37.960119 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:37.961953 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47130.1us
I20260709 13:19:37.959465 21035 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:37.970059 21035 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:37.970430 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.971172 20950 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:37.971906 20950 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:37.972092 21307 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:37.973189 21307 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:37.973526 21307 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:37.961212 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:37.979779 17410 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:19:37.983253 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.983886 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:37.986018 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:37.871245 21293 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:37.986996 21243 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:19:37.987720 21292 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:37.987816 21217 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:37.988252 21383 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:19:37.988317 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 60168us
I20260709 13:19:37.974273 21307 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:37.996802 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.997315 17415 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:38.001554 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.003072 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.003460 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.023566 16591 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:38.003751 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.024500 16596 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:38.017658 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:38.025743 16670 raft_consensus.cc:606] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:38.026140 16670 raft_consensus.cc:614] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:38.030339 16587 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:38.032801 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:37.860051 21236 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.137s
I20260709 13:19:38.038443 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:38.037457 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.043197 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:38.050899 17410 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:19:37.997807 21331 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:38.056030 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.015693 21292 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.015053 21217 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:38.022498 21293 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:38.021179 21383 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:38.066025 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.066219 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.071548 16595 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:38.072407 16588 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:38.053372 21419 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:38.037528 21368 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:38.050350 21429 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.066830 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:38.053049 21230 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:38.078972 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.083529 16587 raft_consensus.cc:2376] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12. Re-sending same reply.
I20260709 13:19:38.064114 21292 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:38.067471 21390 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.086526 16595 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 8.
I20260709 13:19:38.075598 21383 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.050110 21432 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:38.070560 21446 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:19:38.124478 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57737.7us
I20260709 13:19:38.076251 21305 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:19:38.127254 21368 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.092092 21419 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:38.091971 21413 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.126513 21230 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:38.205560 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.206794 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:38.193886 21353 raft_consensus.cc:993] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:38.204176 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:38.211158 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:38.213595 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:38.224629 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.225521 20943 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:38.226076 20943 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:38.227012 21466 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:19:38.228281 20943 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:38.235435 21465 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:19:38.238966 20943 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:19:38.264344 21460 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:19:38.271356 21230 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.215955 21388 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:38.280053 21468 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:19:38.285763 21462 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 13:19:38.207857 21236 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:38.167773 21293 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.301173 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:38.302270 16738 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:38.297515 21368 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:38.338915 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.258535 21428 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.340372 16662 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:38.306566 21425 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:38.332211 20945 consensus_meta.cc:287] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.003s
I20260709 13:19:38.353446 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.385846 16595 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:38.331604 21350 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:19:38.348189 21368 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:38.346823 21425 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.390028 20945 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:38.404578 20945 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:38.405107 20945 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:38.405241 20638 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.621s	user 0.000s	sys 0.007s
I20260709 13:19:38.387092 21419 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:38.409626 16770 heartbeater.cc:646] Failed to heartbeat to 127.15.29.62:37247 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.15.29.62:37247 timed out after 15.000s (SENT)
I20260709 13:19:38.390125 16587 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 4. Current term is 5. Ops: []
I20260709 13:19:38.414673 16662 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:38.395339 21461 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.072s	user 0.001s	sys 0.001s
I20260709 13:19:38.402464 21035 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.432957 21035 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:38.433111 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:38.413374 21452 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.433915 21452 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:38.435436 21464 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:19:38.404100 21428 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:38.439376 21099 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
W20260709 13:19:38.406850 21440 consensus_meta.cc:287] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.591s	user 0.003s	sys 0.005s
I20260709 13:19:38.441037 21385 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Committing config change with OpId 10.9: config changed from index 6 to 9, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:38.443200 21452 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.448361 21307 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:38.448498 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:38.450580 21425 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.340215 21433 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.453014 21433 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:38.452801 17413 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 6 to 9, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:38.400559 21368 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.464615 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:38.465155 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:38.465862 17414 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.446581 21217 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.457346 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:38.458006 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:38.459468 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 244717us
I20260709 13:19:38.460480 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.477766 16595 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:38.465248 16736 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:38.446126 21383 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.494107 16662 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:38.495960 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.509713 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:19:38.513700 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 279618us
I20260709 13:19:38.519021 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:38.521632 20943 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:38.522178 20943 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:38.522692 20943 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:38.521952 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 335303us
I20260709 13:19:38.524978 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:38.525679 21360 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:38.525853 20638 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:38.526139 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.527935 16736 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:19:38.526674 20638 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:38.530643 20638 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.526103 21360 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:38.535224 21363 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:38.536239 21243 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:19:38.535987 21360 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.536746 21243 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:38.537304 21360 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.537678 21243 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.539841 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.548280 16670 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:38.550235 17414 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:38.549880 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:38.556906 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 50135.4us
I20260709 13:19:38.557780 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.511617 21411 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.476130 21035 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:19:38.586196 21433 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:38.518460 21412 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.580947 20945 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.573719 21350 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:38.584985 21236 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:38.587275 21478 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:19:38.589294 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.508354 21292 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.579133 21469 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:38.577818 21431 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:38.508893 21394 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
W20260709 13:19:38.623898 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 270526us
I20260709 13:19:38.631305 21452 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.660382 21452 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.607683 21412 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.662009 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:38.669802 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.674099 16671 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:38.672813 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:38.648398 21431 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:38.686218 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 110283us
I20260709 13:19:38.690353 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:38.632007 21469 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.684202 21292 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:38.699965 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:38.694355 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96290.2us
I20260709 13:19:38.663748 21353 raft_consensus.cc:1081] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:38.694137 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:38.705873 21480 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.709722 21480 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.711118 21440 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:19:38.713017 21480 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.697436 21331 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:38.738761 21473 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:38.748690 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.745038 21331 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:38.759326 21390 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:38.759788 21390 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:38.760484 21388 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:38.762274 21217 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:38.762490 21452 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:38.763212 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.763836 16667 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:38.764137 21469 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:38.764763 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:38.764935 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:38.765744 21448 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:38.766115 21448 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.766453 21448 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:38.760712 21390 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.761317 20945 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.722925 21392 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:19:38.768584 21431 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.777338 16589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:38.778157 16589 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:38.778492 16589 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.780929 16669 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:38.782473 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:38.782619 21412 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.744920 21383 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:38.787313 21293 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:38.758412 21331 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.769414 21392 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:19:38.810213 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.810794 16595 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:38.812618 16634 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:38.813854 21293 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:38.822705 21293 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.819550 16589 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.809242 21225 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.809832 21392 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:38.819820 21481 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.809033 20943 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.861763 21392 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:38.863101 21357 raft_consensus.cc:993] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:19:38.863682 21225 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.865796 21481 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.867359 21392 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:19:38.866622 21225 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:38.872215 21494 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:38.872807 21495 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:19:38.872915 21494 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.875408 21494 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:38.875345 21448 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:38.878329 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65799.2us
I20260709 13:19:38.885579 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.885932 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.886498 16594 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:38.886807 16595 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:38.890408 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:38.891217 16743 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
W20260709 13:19:38.891551 21498 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:19:38.898521 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 189577us
I20260709 13:19:38.892004 21295 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.905659 21295 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.907197 21357 raft_consensus.cc:1081] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:19:38.908366 21295 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.909965 21411 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:38.910394 21411 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:38.911183 21411 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.912443 21411 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:38.912475 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:38.913281 16671 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:38.914407 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:38.915570 21496 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:38.915982 21496 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:38.916329 21496 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:38.917017 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:38.919425 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:38.920146 17414 raft_consensus.cc:2368] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:38.922952 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.923761 16594 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:38.925570 21448 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.924439 16671 raft_consensus.cc:2955] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 6 to 9, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:38.928614 21469 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.929131 21469 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.939136 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.941620 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.941428 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.942308 16588 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:38.942528 16595 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:38.946979 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:38.947353 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:38.948269 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:38.948559 17413 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:38.950798 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.947721 17414 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:38.951275 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.951867 16663 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:38.953811 16636 leader_election.cc:400] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:38.954272 16636 leader_election.cc:403] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:38.939982 16594 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:38.950634 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:38.891882 21360 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:38.962669 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:38.963794 16669 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:38.890287 21481 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:38.965672 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:38.966349 16587 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:38.967150 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:38.967780 16745 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:38.969961 16588 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:38.966715 21295 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:38.972918 17414 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:19:38.973441 17414 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:38.973742 17414 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.971004 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:38.975522 21295 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.975979 21295 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.980808 21449 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:38.981313 21449 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:38.981645 21449 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:38.972510 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:38.985857 16588 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:19:38.935348 20945 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:38.990387 17376 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:38.997558 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:39.003334 21386 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:39.003857 21386 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.004244 21386 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:38.989161 21496 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.008034 21331 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:39.008570 21331 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:39.008884 16589 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.009280 21496 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.009464 21331 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:38.982384 21214 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:38.970656 21360 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.012540 16667 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:39.001438 17414 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.017041 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:39.013609 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:38.994855 20945 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.020846 21392 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:39.021311 21392 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.021675 21392 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.022084 20945 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.031738 16589 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:39.034672 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.038594 17414 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.017884 21360 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.050175 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:39.050536 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:39.051196 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:39.051760 16671 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:39.052003 16671 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:39.052767 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:39.053517 16589 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.006276 21295 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.016871 21505 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:38.993940 21481 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:19:39.063683 16671 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.037611 21385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:52804 (request call id 0) took 2162 ms. Trace:
W20260709 13:19:39.067390 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 94958us
I20260709 13:19:39.068346 16589 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:39.066663 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:39.071105 17413 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:39.071163 16587 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:39.019474 21214 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.064129 21353 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:39.072751 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:39.074656 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:39.063320 21488 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:39.067340 21469 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.090119 21488 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:39.091290 21488 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.082315 21496 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:39.124963 21496 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.125415 21496 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:39.099255 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.099442 16588 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:39.106110 16671 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.111441 16587 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:39.084247 21383 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:39.082610 21353 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:39.122057 21483 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.378s	user 0.001s	sys 0.001s
I20260709 13:19:39.068770 21385 rpcz_store.cc:276] 0709 13:19:36.886387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:36.887300 (+   913us) service_pool.cc:224] Handling call
0709 13:19:36.971432 (+ 84132us) log.cc:844] Serialized 487 byte log entry
0709 13:19:37.635665 (+664233us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b1800235440. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fc20d01 00007f327fb5d691 00007f327fb3deaa 00007f327fb44d53 00007f327fb44cd9 00007f327fb44c69 00007f327fb44c31 00007f327fb43dfd 000000000038f354 000000000038f199
0709 13:19:37.637843 (+  2178us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b1800068b60. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:19:39.037466 (+1399623us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37333,"raft.queue_time_us":38051,"raft.run_cpu_time_us":2054,"raft.run_wall_time_us":125546,"spinlock_wait_cycles":251021312,"thread_start_us":358933,"threads_started":1,"wal-append.queue_time_us":428685,"wal-append.run_cpu_time_us":1667,"wal-append.run_wall_time_us":1018770}
W20260709 13:19:39.138255 21484 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:19:39.143863 16671 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:39.146471 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:39.157364 21386 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.159441 21353 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.151175 21101 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.165354 16589 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:39.140057 21390 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:39.175834 21505 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:39.180685 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:39.174583 21295 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.189677 21101 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.188591 16589 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
W20260709 13:19:39.192874 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 110242us
I20260709 13:19:39.099241 21393 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 13:19:39.198117 16671 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:39.202064 16589 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:39.203691 16596 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:39.188211 21497 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.206024 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 3080) took 1002 ms. Trace:
I20260709 13:19:39.192755 21307 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.188400 21409 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:39.210352 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.193795 21383 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:39.213171 16596 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:39.207530 16742 rpcz_store.cc:276] 0709 13:19:38.203267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:38.203436 (+   169us) service_pool.cc:224] Handling call
0709 13:19:38.203742 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:38.204080 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:38.204108 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:38.207734 (+  3626us) log.cc:844] Serialized 26 byte log entry
0709 13:19:38.452355 (+244621us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b18004bef20. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:19:38.627264 (+174909us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:19:38.627361 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:38.627548 (+   187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:39.121491 (+493943us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:39.205789 (+ 84298us) raft_consensus.cc:1685] finished
0709 13:19:39.205812 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:39.205940 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1675,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":6407,"spinlock_wait_cycles":50610432,"thread_start_us":173857,"threads_started":1,"wal-append.queue_time_us":415588}
I20260709 13:19:39.223596 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:39.225744 21383 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.227578 21505 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.228425 21512 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:39.219069 21292 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.191390 21515 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:19:39.236325 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:39.242950 21515 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:19:39.243378 21515 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:39.243010 21292 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.244170 21515 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.245314 21515 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:39.245424 21292 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.247076 21514 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.247615 21514 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.249707 21514 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.246723 21505 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.235428 21392 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.252449 21505 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.253206 21392 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.258441 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.260619 16596 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:39.263100 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:39.263669 16589 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.262140 16634 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:39.276100 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.217386 21214 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.286899 16589 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:39.212410 21497 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.215515 21307 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.218180 21409 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.285813 21392 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:39.312757 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 101783us
I20260709 13:19:39.315201 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:39.318120 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.318725 16671 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:39.322350 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.325054 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.325834 16668 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:39.304311 20943 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.333920 16670 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:19:39.305744 21392 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.314987 21350 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.348802 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:39.352145 17413 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:39.343096 21392 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.336750 21516 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:39.343900 21101 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.314810 21449 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.340673 21494 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:39.333551 21393 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:19:39.350850 21295 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.378412 21514 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.378826 21386 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.365512 21516 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.383232 21516 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.386238 16589 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:39.386693 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.389020 17411 raft_consensus.cc:2368] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12.
I20260709 13:19:39.355650 21292 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:19:39.388940 21353 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.392820 21497 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.377305 21214 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:19:39.376696 21494 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.385859 21230 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.385119 21409 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.406147 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:39.358704 21350 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.441818 21497 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.409909 21101 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.452905 21360 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.456684 21307 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.457772 16662 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:19:39.426918 21469 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:19:39.406795 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:39.410125 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.426795 21293 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.458411 17413 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:39.472317 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:39.440603 21432 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.471738 16589 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
W20260709 13:19:39.437150 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:39.447732 16666 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:39.463630 21360 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.464368 21449 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.465595 21101 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.466324 21497 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.467569 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.479053 16664 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:39.379022 21514 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.479364 21432 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.492483 21514 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.493538 21432 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.470286 21035 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.493896 21392 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.494268 21035 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.444382 21230 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.468572 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.477101 16663 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:39.377797 21295 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.480456 21305 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:39.481796 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.435515 21353 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.478839 21293 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.505334 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:39.506074 17413 raft_consensus.cc:2368] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:39.506223 21353 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.506335 21419 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.506986 21419 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.483784 21497 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.508526 21295 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.506587 21293 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.508760 21494 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.509613 21419 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.511830 21514 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:39.515123 21305 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.517428 21305 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.518601 16669 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:39.485502 21350 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.486572 16666 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:39.488837 21496 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.485496 21101 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.495149 16671 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:39.502563 16665 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:19:39.477272 20943 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:39.520851 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.521727 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.534669 16587 raft_consensus.cc:2368] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:39.527187 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.529098 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.531579 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:39.531736 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.532059 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.537369 16736 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:39.537930 16634 leader_election.cc:400] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:39.538228 16634 leader_election.cc:403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:39.534782 16664 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:39.539448 16589 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:19:39.540082 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:39.540779 17415 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:39.544014 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.527318 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.517964 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:39.545415 16668 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:39.532418 16666 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:39.501947 21214 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.546984 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.521173 21353 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:39.546036 20943 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.560554 20945 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:39.561136 20945 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.563815 16665 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:39.553085 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.570535 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.571203 16739 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.573148 16736 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:19:39.534224 21419 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.533428 21385 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:39.592327 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.592525 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.597189 16669 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:39.592742 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.597657 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.597819 16662 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:39.602197 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:39.603166 17415 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:39.598342 16745 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:39.495952 21516 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.555977 21101 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:39.558635 21497 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:19:39.566792 21350 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:39.622953 16738 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:39.563683 21353 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.612249 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.566457 21214 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.632967 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.618037 21101 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.590783 21469 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:39.625002 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:39.625649 16738 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73089175ee9c465197a1a11e304346be"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:19:39.635721 16596 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:39.627125 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.628028 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.599325 21360 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:39.605971 21447 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.622771 21425 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.658488 21425 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.661190 21425 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.663094 20945 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.666246 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.608335 21385 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.582418 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.632229 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.668977 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.632977 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:39.669468 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 87160.6us
I20260709 13:19:39.619618 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:39.606276 21293 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:19:39.672770 21516 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.675294 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.675979 16740 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:19:39.673043 20945 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:39.677428 20945 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:39.649569 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.679531 16736 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.678485 20945 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.683784 21035 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.651140 16594 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:19:39.688447 21230 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.690924 21214 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.693248 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.694689 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.695346 16741 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:39.636790 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.652346 16745 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:19:39.631526 16592 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:19:39.631976 21353 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.630609 21350 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.655776 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:39.615592 21449 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.703192 21449 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.704253 21496 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:39.649782 21469 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.705204 21449 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:39.670320 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50785us
I20260709 13:19:39.670347 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.676854 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.707487 16740 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:39.696105 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.605684 21419 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.696273 16739 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:39.708425 16737 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:39.636157 21101 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:39.698820 16588 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:39.700016 16736 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:39.701769 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.696954 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.709086 16738 raft_consensus.cc:606] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Received request to transfer leadership
I20260709 13:19:39.702595 21392 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:39.709730 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54046.9us
I20260709 13:19:39.711802 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.713136 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:39.712507 16745 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:39.712546 16738 raft_consensus.cc:614] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:19:39.714102 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43868us
I20260709 13:19:39.714699 21525 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:39.714995 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:39.715553 17415 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:39.716003 21419 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.657208 21360 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.716470 21214 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:39.717226 21528 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:39.718022 21385 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.705825 21350 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:39.719585 21214 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:39.719938 21214 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:39.698973 21393 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:39.718091 21360 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.721395 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:39.746646 17415 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:19:39.668308 21447 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.725752 21393 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.852018 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.711045 21449 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:39.777580 21533 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:19:39.722347 21390 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:19:39.827901 21293 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:19:39.828662 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 106870us
I20260709 13:19:39.711277 21496 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.727011 21419 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:19:39.860443 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.861052 16595 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:39.858788 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.866159 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.868166 21425 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:39.868743 21425 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.869164 21425 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:39.872587 21293 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:39.873054 21293 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.873674 21293 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.875584 21419 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:39.876017 21419 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:39.876912 21419 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.878181 21419 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:39.854777 21393 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.835415 21471 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:39.836691 21526 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.838486 21392 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.839031 16670 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:39.891114 16670 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:39.892009 16670 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.840488 21447 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.840353 21101 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.842826 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:39.896919 16592 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.844684 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:39.846395 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.849555 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.900537 16668 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:19:39.850494 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.901620 16665 raft_consensus.cc:2410] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:19:39.851157 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.828027 21214 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.853981 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.856612 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.902257 16740 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:39.859901 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.902269 16745 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:39.855453 21528 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:39.914742 16592 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:39.917613 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:39.924502 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.916051 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.833678 21497 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:39.888458 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:39.889871 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.893285 16670 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:39.936398 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.857802 21390 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.905632 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.905802 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:39.928493 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:39.938530 16744 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:39.866431 21496 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.939551 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:39.908000 21307 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:39.946090 16670 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:39.945920 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.947994 16744 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:39.923046 21217 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:39.925509 21035 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:19:39.922987 21528 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.949827 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:39.954595 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.954584 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:39.896517 21392 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.942718 21518 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:39.959879 16738 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:19:39.963498 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:39.918783 21293 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:39.967928 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.942618 21390 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:39.972391 16736 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:39.972921 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.898692 21526 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.896981 21471 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:39.956329 21388 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:19:39.950465 21214 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:39.971364 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:39.978622 16738 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:39.975199 16740 raft_consensus.cc:2368] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:39.980547 21392 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:39.981541 21534 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:19:39.972483 21230 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:39.983381 21230 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:39.983769 21230 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:39.984586 21526 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:39.986373 21293 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.959187 21516 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:39.965929 21528 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:39.953128 21307 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:39.975864 21525 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:39.959945 21433 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:19:39.960458 21217 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.083822 21217 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.058291 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:39.973695 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:40.087553 16662 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:40.087921 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:40.089085 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 115301us
I20260709 13:19:40.090597 21035 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:40.091245 21035 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.091369 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:40.094803 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:40.073318 21101 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.105454 21101 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:40.107132 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:40.107690 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.108300 17414 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.109248 17415 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 10: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:40.109905 21305 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:40.111230 21305 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:40.072345 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:40.077342 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.115622 16742 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:40.115015 21101 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.115775 21307 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.116907 20638 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.117741 20638 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.118369 21350 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.076280 16741 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:40.070514 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:40.096463 20945 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.089108 16663 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.124361 16736 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:40.091842 21035 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:39.996927 21469 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:39.989254 21516 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:40.000887 21525 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:40.008818 21535 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.353s	user 0.001s	sys 0.002s
I20260709 13:19:40.167912 21516 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:40.158121 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:40.133445 21350 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:40.170259 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64295.7us
I20260709 13:19:40.175275 21433 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:40.134963 21496 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.176319 21525 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.182265 21469 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:40.160158 20945 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:40.182998 21469 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.154341 20638 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:40.184007 20945 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.185503 21469 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.156750 21471 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.190330 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.190769 16636 leader_election.cc:400] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:40.191385 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:40.192989 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.193187 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.193821 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:40.195631 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.183907 21101 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.197825 21392 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:40.198975 21392 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.200229 16670 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:40.201817 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.202154 21392 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.216356 21471 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:40.225221 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.226622 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.228353 21446 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:40.228797 21446 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:40.229223 21446 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:40.231149 21532 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:40.231674 21532 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:40.232141 21532 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:40.208254 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.244843 17414 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:19:40.210166 21230 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.246112 16736 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:19:40.233695 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:40.260795 21471 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:40.264849 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:40.266039 21471 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:40.273880 21353 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.006s
I20260709 13:19:40.274789 21230 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:40.276401 16744 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:19:40.281767 16664 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:40.290304 16666 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.277137 21353 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.300160 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.304804 21353 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:40.306036 16671 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:40.307497 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.308696 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:40.309181 17415 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:19:40.309593 17415 pending_rounds.cc:85] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 8
I20260709 13:19:40.309805 21452 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:40.310316 21452 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.310438 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.309279 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.310760 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.312511 17413 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:19:40.312577 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.310758 21452 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:40.313215 17410 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:40.317543 16669 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:19:40.309897 17415 pending_rounds.cc:107] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:19:40.319825 16663 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:40.319275 17415 raft_consensus.cc:2889] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:40.333527 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.331328 16636 leader_election.cc:400] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12.
I20260709 13:19:40.333498 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.335772 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:40.338428 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.343281 21526 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:40.344210 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:40.344971 16588 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:40.352351 20943 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.355873 21526 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:40.357326 21526 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.333170 21530 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:40.389552 21425 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.005s
I20260709 13:19:40.393010 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:40.393585 21425 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.397974 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:40.393059 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 72273.3us
I20260709 13:19:40.400614 21419 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:40.401379 21419 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:40.408715 21393 consensus_meta.cc:287] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.503s	user 0.016s	sys 0.002s
I20260709 13:19:40.412985 21388 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:40.413779 21553 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:19:40.413930 21388 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.371376 21530 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.417410 21530 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:40.419732 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.392427 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:40.416793 21388 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.422744 16588 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:19:40.410995 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.426152 16595 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:40.342394 20945 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:40.434842 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 140802us
I20260709 13:19:40.433130 17411 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
W20260709 13:19:40.439793 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:40.404585 21243 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:19:40.377100 21390 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.438333 21353 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.445523 21353 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.448352 21353 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:40.457777 16594 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:40.433967 21388 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.463945 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.467253 16664 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:40.476154 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.479935 21419 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.473877 21452 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.475342 21446 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.377307 21516 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.479887 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.491792 16664 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:40.472273 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.410637 21393 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.482856 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.483282 16595 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:40.483922 16666 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
W20260709 13:19:40.486203 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49489.5us
I20260709 13:19:40.479894 16668 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:40.488973 21425 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:40.489240 16667 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:19:40.496093 16592 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:19:40.478276 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.444285 21471 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:40.503692 17419 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:40.509009 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.509197 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.512519 16743 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:40.513083 21230 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:40.513465 16634 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.482630 21035 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.514467 16595 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 5.
I20260709 13:19:40.514592 16588 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:40.511766 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:40.513777 21230 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.495034 21388 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.518666 16668 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:40.519201 21353 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:40.498070 20945 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.510840 21471 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:40.523335 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.523576 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.524456 16588 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:40.522920 21392 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:40.525242 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.525573 21392 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:40.526391 21452 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:40.527769 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:40.519758 21446 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.530805 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.529465 16593 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:40.525602 21385 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:40.523082 21353 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:40.534992 21496 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:40.526787 21392 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.536741 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.537498 21496 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.537708 21469 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:19:40.539057 21385 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:40.546092 21528 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.549s	user 0.017s	sys 0.000s
I20260709 13:19:40.546332 21214 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:40.542481 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:40.547534 21385 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:40.486912 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:40.548151 16595 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:40.548190 21035 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.550171 21471 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.551719 21516 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:40.553282 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:40.553740 16488 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 143) took 13935 ms (client timeout 15000 ms). Trace:
I20260709 13:19:40.554163 16744 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:40.545528 21393 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:40.559856 20945 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:40.544813 21469 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.560828 21469 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:40.561919 21528 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.559890 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.562600 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.564072 21528 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:40.564463 16592 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:40.565142 16745 raft_consensus.cc:2368] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14.
I20260709 13:19:40.546303 21532 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.544973 21496 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.566828 21388 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.567874 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.576496 16667 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:40.568032 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.569536 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.578140 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:40.578778 17415 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.543063 21353 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.575965 21388 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:40.568449 21230 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.567031 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.582737 16668 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.571048 17419 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:40.584293 16594 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:40.592315 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.593562 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:40.594188 21555 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:19:40.594301 16587 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.594570 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.595108 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.569792 21532 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.600476 17415 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:40.603634 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:40.604504 21558 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:19:40.605393 21419 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:40.606081 21419 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:40.607196 21419 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.612474 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:40.615368 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.615976 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:40.562577 21530 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.624505 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:40.625258 16744 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:40.554256 16488 rpcz_store.cc:269] 0709 13:19:26.617699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:26.617947 (+   248us) service_pool.cc:224] Handling call
0709 13:19:40.553477 (+13935530us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:38.164189 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:38.331305 (+167116us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:38.337652 (+  6347us) write_op.cc:435] Acquired schema lock
0709 13:19:38.341661 (+  4009us) tablet.cc:662] Decoding operations
0709 13:19:38.441734 (+100073us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:38.441792 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:38.441819 (+    27us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:19:38.555798 (+113979us) tablet.cc:701] Row locks acquired
0709 13:19:38.573356 (+ 17558us) write_op.cc:260] PREPARE: finished
0709 13:19:38.579672 (+  6316us) write_op.cc:270] Start()
0709 13:19:38.579805 (+   133us) write_op.cc:276] Timestamp: P: 1783603178579616 usec, L: 0
0709 13:19:38.579831 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:19:38.697280 (+117449us) log.cc:844] Serialized 6392 byte log entry
0709 13:19:38.901362 (+204082us) op_driver.cc:464] REPLICATION: finished
0709 13:19:39.087942 (+186580us) write_op.cc:301] APPLY: starting
0709 13:19:39.094192 (+  6250us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:39.226354 (+132162us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:39.226457 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:39.441541 (+215084us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:39.455118 (+ 13577us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:19:39.455173 (+    55us) write_op.cc:312] APPLY: finished
0709 13:19:39.455729 (+   556us) log.cc:844] Serialized 148 byte log entry
0709 13:19:39.455913 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.456069 (+   156us) write_op.cc:454] Released schema lock
0709 13:19:39.456126 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:39.456525 (+   399us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:39.456557 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":107791,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":174356,"apply.run_cpu_time_us":24192,"apply.run_wall_time_us":372754,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71071,"num_ops":16,"prepare.queue_time_us":163909,"prepare.run_cpu_time_us":16970,"prepare.run_wall_time_us":671719,"raft.queue_time_us":29385,"raft.run_cpu_time_us":26493,"raft.run_wall_time_us":138480,"replication_time_us":321308,"thread_start_us":288985,"threads_started":4,"wal-append.queue_time_us":151938,"wal-append.run_cpu_time_us":9552,"wal-append.run_wall_time_us":1594864}]]}
I20260709 13:19:40.631806 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.616019 21452 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:40.633929 21452 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:40.635175 21452 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.637811 16595 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:40.639053 21530 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.618635 21230 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 13:19:40.663321 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 126415us
I20260709 13:19:40.651783 21350 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:19:40.675616 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.661792 21392 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:40.688176 16745 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:19:40.690464 16745 pending_rounds.cc:85] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 8
I20260709 13:19:40.690814 16745 pending_rounds.cc:107] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 15.9
W20260709 13:19:40.687402 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 150719us
I20260709 13:19:40.691102 16745 raft_consensus.cc:2889] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:40.672549 21350 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:40.698916 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50279.1us
I20260709 13:19:40.702924 21350 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:40.690377 21392 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:40.744612 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.722980 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:40.750427 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 130956us
W20260709 13:19:40.751523 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 135598us
I20260709 13:19:40.757748 16744 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:40.760049 20945 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.760802 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:40.760748 20945 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.761272 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.761619 16740 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.726994 21524 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:40.763769 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:40.767439 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 90999.4us
I20260709 13:19:40.777709 16587 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:19:40.779754 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:40.722800 21390 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:40.783262 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:40.784612 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.791167 16739 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.791191 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.795011 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:40.796089 20945 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.757176 21230 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:40.762804 21524 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:40.806095 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53687.9us
I20260709 13:19:40.809298 16739 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:19:40.811332 21565 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:19:40.813344 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.814085 16671 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:40.820677 21496 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.823546 21559 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:40.824388 21559 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:40.824590 16542 auto_rebalancer.cc:893] Removing replace marker failed: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:40.825098 16542 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:19:40.825398 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:19:40.825438 21559 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.830072 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:40.834018 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:40.835379 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.836064 16664 raft_consensus.cc:2368] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:40.836161 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.836766 16666 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:40.839021 16664 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:19:40.838982 21230 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:40.839764 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.807982 21524 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.840467 21230 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.846912 16587 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:40.813565 21392 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.839298 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.857787 16665 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:40.858767 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:40.858754 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 75544.5us
I20260709 13:19:40.861251 16668 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:40.855844 16666 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
W20260709 13:19:40.866477 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79919.1us
I20260709 13:19:40.869681 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:40.873373 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69492us
I20260709 13:19:40.872535 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:40.875195 16742 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 4.
I20260709 13:19:40.875427 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:40.882686 21293 raft_consensus.cc:2764] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:19:40.867642 21425 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:40.883262 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:40.883755 21425 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.885667 21392 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:40.877336 16664 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:40.900580 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:40.907995 16665 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:19:40.919212 16671 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:19:40.922973 21519 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:40.923378 21243 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:40.931214 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:40.932572 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:40.933336 21293 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:40.933705 21101 raft_consensus.cc:2764] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:19:40.937391 16740 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:19:40.938383 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:40.937517 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:40.964327 21390 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:40.965119 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:40.965226 21243 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:40.967015 21519 raft_consensus.cc:2764] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:19:40.968200 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:40.970592 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.961011 21496 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.971180 21452 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:40.971729 21452 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:40.972167 21452 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:40.972656 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:40.946326 21101 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:19:40.974234 21101 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:40.974601 21101 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:40.974766 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:40.975663 17410 raft_consensus.cc:2368] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:40.975723 21101 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:40.975667 21532 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:40.982808 16490 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 9 to 11, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:19:40.987553 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69500.7us
I20260709 13:19:40.994127 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:40.996097 21425 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.996716 21350 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:40.999248 21350 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:40.946689 21469 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.000692 16662 raft_consensus.cc:2368] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:19:41.003932 21469 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:41.008001 21524 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.009284 17411 raft_consensus.cc:2368] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:41.020316 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:41.015964 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:41.025018 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62509.7us
I20260709 13:19:41.010152 16490 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 8, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.027318 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.030328 16593 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:19:41.029762 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.029175 21393 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:41.041177 21393 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:41.041608 21393 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:41.045092 21446 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:41.045897 21446 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:41.058851 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:41.060300 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.049798 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:41.062549 16671 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:41.046389 21452 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.101868 17410 raft_consensus.cc:2368] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:41.102998 21524 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.066351 21528 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:41.085044 21101 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:41.087733 16744 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:41.088363 16559 leader_election.cc:400] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:41.108925 16593 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:19:41.095144 21446 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.109432 21524 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:41.109108 16559 leader_election.cc:403] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.112840 21532 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:41.088902 21390 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:41.116532 21390 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:41.117486 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:41.118263 21293 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:41.118597 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:41.118768 21293 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:41.119709 21293 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.120944 20945 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:19:41.121490 20945 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:41.122277 20945 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:41.125950 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 73332.1us
I20260709 13:19:41.126708 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:41.129624 21471 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:41.130205 21471 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.132272 21471 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:41.133136 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:41.134296 21471 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:41.134654 16561 leader_election.cc:400] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:41.136292 16561 leader_election.cc:400] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:41.136556 16561 leader_election.cc:403] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.137226 21390 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:41.139031 21471 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:41.139124 21452 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.139667 21471 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.143996 21471 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:41.063652 21392 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.146500 21392 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.149760 21390 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15.
I20260709 13:19:41.137231 16561 leader_election.cc:400] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:41.159052 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:41.159803 16587 raft_consensus.cc:2368] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:19:41.165609 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.166290 16588 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:41.166618 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.167949 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.170250 16596 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:41.158160 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:41.174342 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47751.4us
W20260709 13:19:41.175625 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 71387.9us
I20260709 13:19:41.177938 16592 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:41.180279 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:41.181555 16588 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:41.176229 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:41.185221 16708 leader_election.cc:400] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:19:41.185712 16708 leader_election.cc:403] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:41.186355 21390 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:41.193979 21214 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:19:41.209671 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.212627 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.229645 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.172968 21452 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.233215 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:41.186470 21496 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:19:41.236486 16671 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:41.237390 16670 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:41.240343 21518 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.241567 21385 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.197710 21307 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:41.198555 21469 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:41.242589 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:41.244437 16490 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 4 to 11, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.254338 21230 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.265129 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.265375 21230 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.267987 21230 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:41.270296 16490 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 9, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 9 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.259198 21392 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:41.259384 21528 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.250227 21385 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.259784 16595 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:41.291775 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:41.292647 16592 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:41.292733 16490 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.260090 21307 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:41.244828 21419 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.260499 21496 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:41.310954 16595 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:41.281837 21518 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.313138 21573 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:41.292070 21469 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:41.315140 21473 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:41.316083 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:41.316804 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:41.317497 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:41.317853 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:41.352007 16742 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:19:41.318744 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:41.320150 16743 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:19:41.320775 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 121120us
I20260709 13:19:41.308723 21307 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:41.343266 16490 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 11, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 11 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:41.372673 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 113649us
W20260709 13:19:41.373445 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 84666.8us
I20260709 13:19:41.373838 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:41.289868 21528 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.290460 21452 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.390372 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:41.396457 16592 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:41.308171 21516 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:19:41.400434 16595 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:41.419045 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:41.373600 21390 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:41.432943 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:41.437429 16669 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:41.433802 16743 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:41.441826 16490 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.447232 16490 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.381599 21419 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:41.468586 21584 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:19:41.469589 21590 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.470172 21590 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.472425 21590 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.474843 21307 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.476755 16588 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:41.478343 21452 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:41.479126 16490 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:41.479796 21518 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.481393 21419 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.481002 16490 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 2 to 9, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 9 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:41.481910 21413 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.482460 21413 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.484539 21413 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:41.486335 21385 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:41.489466 21528 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.490015 16587 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:19:41.490350 21390 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.491142 21588 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:41.496668 17410 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
W20260709 13:19:41.501020 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 100219us
I20260709 13:19:41.510699 21532 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11.
I20260709 13:19:41.530015 21385 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:41.530828 21597 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:41.536237 21526 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:41.536717 21526 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:19:41.537137 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 104448us
I20260709 13:19:41.537868 16594 raft_consensus.cc:2410] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:19:41.539666 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:41.540980 21385 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.544090 21385 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.544371 21307 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:41.527448 21469 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:41.550209 21578 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:19:41.555763 21573 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:41.556298 21573 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.556737 21573 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:41.557796 21568 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.558432 21568 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.449258 21496 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.576839 16589 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:41.579655 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:41.580773 21214 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:19:41.581161 16636 leader_election.cc:400] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15.
I20260709 13:19:41.575178 16490 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 8, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.584625 16587 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:41.587299 21350 raft_consensus.cc:2764] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:19:41.590175 21572 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:41.591591 21214 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:41.592612 16708 leader_election.cc:400] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 5. Current term is 6.
I20260709 13:19:41.592326 21214 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.593164 16591 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:19:41.593611 21392 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:41.593824 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:41.581048 21393 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.001s
W20260709 13:19:41.608224 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 225362us
I20260709 13:19:41.537675 21526 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.613881 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:41.594789 21214 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.598802 21589 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:41.623927 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:41.624914 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:41.626044 16742 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:19:41.627156 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:41.665336 16669 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:41.672724 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.675102 17411 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:41.609238 21393 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:41.680214 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.627419 21516 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:41.677836 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.619405 21452 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:19:41.677721 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:41.687383 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 68522.1us
I20260709 13:19:41.688319 16490 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.690115 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.690205 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.691699 16596 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:19:41.619823 21413 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:19:41.693264 16666 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:41.694052 16671 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:41.669150 21353 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:41.695818 21582 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
I20260709 13:19:41.619552 21590 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:19:41.704360 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:41.694593 21393 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:41.669039 21385 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:41.711185 21599 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:41.715627 21583 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.269s	user 0.002s	sys 0.000s
I20260709 13:19:41.676254 21419 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:41.726281 21419 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.727383 16668 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:41.733599 21568 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:41.734876 21413 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:41.735673 21581 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.268s	user 0.002s	sys 0.000s
I20260709 13:19:41.735479 21413 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:41.771335 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 70112.7us
I20260709 13:19:41.699905 21526 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:41.815953 16490 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.838829 21413 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:41.814556 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 142667us
I20260709 13:19:41.834942 16490 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 16 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.951618 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:41.938762 16490 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:41.810401 21385 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:41.711896 21599 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.988063 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:41.990724 21604 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.991390 21604 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.996563 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:41.663789 21589 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:41.926062 21532 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:41.998272 21532 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.000358 21532 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.005555 21573 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.009142 16669 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:19:41.948760 21392 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:41.950320 21419 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:41.953681 21393 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:42.028467 16490 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 10, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:42.047665 16490 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:42.049525 16744 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:42.050853 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:42.051633 16669 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:42.056770 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.072762 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.073496 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.074345 16742 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:19:42.078037 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:42.058126 21425 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:42.080864 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 283118us
I20260709 13:19:42.044865 21393 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.046589 21392 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.085038 21596 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:42.085557 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:42.093035 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:42.096817 16669 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:42.093617 21214 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:42.124370 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:42.107599 21608 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:42.117093 21419 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:42.126449 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.011s	sys 0.016s
W20260709 13:19:42.134634 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.011s	sys 0.016s
W20260709 13:19:42.134987 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 622463us
I20260709 13:19:42.123685 21214 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.092269 21425 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.139611 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:42.116626 21590 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:19:42.126987 21608 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.149807 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:42.169919 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51681.3us
I20260709 13:19:42.173825 21393 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.127518 21573 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.138232 21419 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.089262 21385 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.137702 21592 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:19:42.142031 21214 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.103323 21596 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.170637 21604 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.142282 21350 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:42.171499 21532 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:42.185406 21293 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:19:42.199820 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 67076.6us
I20260709 13:19:42.175426 21608 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:42.220939 16594 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:19:42.224287 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 116382us
I20260709 13:19:42.233173 16636 leader_election.cc:400] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14.
I20260709 13:19:42.186501 21419 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:42.229741 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:42.242007 21392 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.242976 21392 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:42.247924 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:42.249137 21447 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:19:42.250128 21447 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:42.250444 21612 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:42.250510 21447 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:42.251026 21612 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:42.251106 21447 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.252020 21612 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.253245 21612 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:42.255463 21606 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 13:19:42.255905 16594 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:42.258307 21496 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:42.260288 21496 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:42.260663 21496 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:42.261459 21496 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.262583 21496 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:42.265658 21608 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.268486 21608 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.273240 21597 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.279121 21615 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.279949 21615 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.287875 21605 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.288736 21605 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.215215 21599 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.293243 21447 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:42.227011 21600 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.412s	user 0.003s	sys 0.000s
I20260709 13:19:42.296655 21425 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.227233 21293 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:42.301257 21293 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.311086 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:42.311877 17415 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:42.315029 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 141353us
I20260709 13:19:42.234488 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:42.335253 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 149598us
I20260709 13:19:42.340472 21596 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:42.344537 21526 consensus_meta.cc:287] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.021s
I20260709 13:19:42.363653 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.364337 21390 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.364845 21390 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.365221 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.366743 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.367034 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.367453 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:42.367861 21589 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.368772 21390 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.369336 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.371496 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.372084 16593 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:42.373814 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.378353 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.274101 21597 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.336299 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:42.415733 17415 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
W20260709 13:19:42.432010 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 115796us
I20260709 13:19:42.432802 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:19:42.436794 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 112682us
I20260709 13:19:42.445917 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.446022 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:42.448652 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 112439us
W20260709 13:19:42.430549 21610 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.208s	user 0.005s	sys 0.009s
I20260709 13:19:42.454700 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.455312 17414 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:19:42.430650 21607 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.282s	user 0.004s	sys 0.006s
W20260709 13:19:42.429966 21606 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.163s	user 0.009s	sys 0.014s
I20260709 13:19:42.464663 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.466919 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.458549 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:42.469679 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.470273 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.472340 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:42.473449 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 239068us
I20260709 13:19:42.472343 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:42.479245 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.479853 16671 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:42.480746 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1681) took 1165 ms. Trace:
I20260709 13:19:42.481640 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:42.483130 21613 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.132s	user 0.001s	sys 0.011s
I20260709 13:19:42.482841 21293 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.483336 21559 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:42.483732 21559 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:42.484514 21559 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.485044 21293 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.485724 21559 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:42.491532 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:42.492154 16669 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:42.482990 21595 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.493180 16739 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:42.478853 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:42.496762 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.497741 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:42.498433 16667 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:42.491154 16596 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:42.454944 21353 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.455157 21518 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:42.457700 21452 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:42.481165 16737 rpcz_store.cc:276] 0709 13:19:41.314802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:41.318952 (+  4150us) service_pool.cc:224] Handling call
0709 13:19:41.319273 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:41.371603 (+ 52330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:41.371637 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:41.372345 (+   708us) log.cc:844] Serialized 26 byte log entry
0709 13:19:41.598621 (+226276us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:19:41.598667 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:41.598967 (+   300us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:42.115630 (+516663us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:19:42.471074 (+355444us) raft_consensus.cc:1685] finished
0709 13:19:42.471094 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:42.480662 (+  9568us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131674,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":7879,"spinlock_wait_cycles":20309376,"thread_start_us":251647,"threads_started":2,"wal-append.queue_time_us":303037}
I20260709 13:19:42.469310 21496 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.468823 21392 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:42.514644 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.516765 16671 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:42.517951 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:42.496382 21469 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.536974 16664 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:42.539171 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.503340 21618 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:42.536468 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.511037 21350 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.544754 16744 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:42.543239 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.511468 21447 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.573673 16664 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:19:42.575471 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:42.579653 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 107394us
I20260709 13:19:42.588291 21616 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.588596 21597 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.588943 21616 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.535387 21608 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:42.535599 21496 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.591194 21608 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.593222 21496 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.593389 21608 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.528586 21353 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.581311 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:42.598786 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:19:42.602319 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.610193 16662 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:42.615190 16670 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:42.504408 21595 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.573622 21612 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:42.575183 21392 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:42.573182 21615 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:42.636929 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 158289us
I20260709 13:19:42.638276 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:42.632511 16668 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:42.607574 21618 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:42.632150 16669 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:42.543869 21518 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.648144 17415 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:19:42.639691 16667 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:42.624982 21447 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:42.657442 21629 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:42.663761 21618 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.669493 21214 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.581135 21469 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:42.679602 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.682664 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85161.2us
I20260709 13:19:42.635726 21588 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:42.600819 21350 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.709380 16736 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:42.713318 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:42.715653 16737 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:42.652107 21612 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:42.722393 16743 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:42.691402 21587 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:42.702023 21620 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:42.670889 21385 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 13:19:42.729797 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 91646.9us
I20260709 13:19:42.719892 21608 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.721704 21392 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.732734 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:42.721455 21447 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:42.721400 21452 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:42.736753 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:42.737833 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:42.739090 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:42.743458 21637 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.745453 21573 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:42.734555 21620 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.752259 21214 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.736024 21608 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.767727 20638 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:42.774649 21630 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:42.756408 21350 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.756939 21631 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:42.757146 21413 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:42.760259 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:42.764516 21595 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.765129 21612 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.774822 20638 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:42.835690 21626 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 13:19:42.837381 20638 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.766304 21447 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.768797 16737 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:19:42.776000 21608 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:42.839373 21609 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
I20260709 13:19:42.840601 21353 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:42.844395 21617 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:42.846398 21624 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:19:42.847424 21625 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:19:42.744802 21616 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.849834 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.772162 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:42.832623 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 72438.3us
I20260709 13:19:42.730856 21587 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.860299 21587 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.860726 21518 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.738315 21392 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.861780 21469 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:42.791019 21419 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.002s
I20260709 13:19:42.775683 21630 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:42.876184 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 104124us
I20260709 13:19:42.877707 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:42.878446 16667 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:42.905320 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 382961us
I20260709 13:19:42.908049 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:42.909080 16668 raft_consensus.cc:2410] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:19:42.862528 21452 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.885078 21518 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:42.911854 16670 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:42.917255 21634 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:42.917279 21621 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.897222 21353 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.865722 21419 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.874081 21587 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:19:42.909298 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 2.099s (SENT) [suppressed 1 similar messages]
I20260709 13:19:42.773689 21637 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.942412 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:19:42.910065 21413 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:42.857100 21631 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:42.945329 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 80157.1us
W20260709 13:19:42.962402 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54280.4us
I20260709 13:19:42.966655 16742 raft_consensus.cc:2368] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:42.967243 16667 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:19:42.954639 21392 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:42.979081 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.979903 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:42.981583 16595 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:42.982115 16589 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:42.983227 16744 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:19:42.891757 21630 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:42.946812 21518 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.977979 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:42.986353 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:42.987149 16668 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:42.989283 21559 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.503s	user 0.014s	sys 0.000s
I20260709 13:19:42.989701 21605 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:42.990161 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.986050 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.991430 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:42.991823 16662 raft_consensus.cc:2410] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:19:42.992959 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:42.952525 21621 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:42.996405 21645 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:19:43.018759 21596 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:42.966683 21293 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:19:42.961136 21353 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.020502 16669 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:19:43.020902 16669 pending_rounds.cc:85] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 9
I20260709 13:19:43.021118 16669 pending_rounds.cc:107] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:19:43.021318 16669 raft_consensus.cc:2889] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:43.022132 21353 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:43.022771 16670 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:19:43.000722 21419 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.002063 21620 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.002240 21631 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:43.003999 21621 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.004210 21524 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.025720 21524 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:42.964367 21413 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.026623 21524 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.027765 21524 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.028035 16671 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:43.031178 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.004019 21637 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:43.008677 21643 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 13:19:43.009049 21630 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:43.010093 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.011157 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.011812 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:43.013067 21518 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.015988 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:43.016539 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.017879 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.055840 16666 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:19:43.055958 16667 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:19:43.018067 21496 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:43.058908 21617 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:43.059007 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:43.059602 21617 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:43.060585 21617 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.032001 16662 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
W20260709 13:19:43.063786 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49784.8us
I20260709 13:19:43.064198 21587 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:43.064638 21587 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:43.065398 21587 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.054083 16738 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:43.067795 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.063818 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.068826 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.070104 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.072077 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.072861 16742 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:43.073915 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:43.057564 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.083640 21413 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.088920 21630 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:43.089561 21630 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.091697 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.093719 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.057031 16736 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:43.069145 16741 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:43.068898 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.097321 16743 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:43.100008 16636 leader_election.cc:400] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11.
I20260709 13:19:43.104977 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.106850 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.114257 16708 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.116743 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:43.118132 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.119733 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.117853 16739 raft_consensus.cc:2410] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:19:43.125608 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.127377 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:43.127897 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.130577 16737 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 6.
I20260709 13:19:43.109380 21599 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:19:43.133109 21587 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.136335 21650 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:43.136933 21650 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.108815 21637 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.145221 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.146955 21630 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.148588 16739 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:43.151279 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.151670 21573 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:43.153010 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.153892 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.154793 21622 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:43.153015 21573 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.155172 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.156646 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:43.159107 16589 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:43.163682 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.165243 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.165815 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.163727 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.112982 21393 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:43.059078 21496 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.157755 21419 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:19:43.190301 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.160000 21604 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:43.215966 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.216413 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.217211 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.214609 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.168322 21573 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:43.161693 21637 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:43.168067 21599 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:43.167572 21615 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.168809 21528 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.167179 21644 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.225121 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:43.169523 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 260245us
I20260709 13:19:43.206472 21524 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:43.226272 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.228267 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.229732 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.231654 16668 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:43.204947 21630 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:43.236389 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.237119 16662 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:43.235365 21524 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.239224 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.158414 16663 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:43.241098 16663 raft_consensus.cc:606] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.241160 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.225867 21350 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:43.243945 16665 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:43.243906 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.220566 21353 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:19:43.244146 21350 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:43.244715 16741 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:43.244891 21350 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:43.245041 21353 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:43.245594 21353 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:43.246481 21353 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.247967 21353 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:43.239226 21524 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.222458 21611 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:43.222446 21650 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.230636 21655 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:19:43.255492 21350 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.260980 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.261544 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.262279 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.262660 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.263305 16595 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 7.
I20260709 13:19:43.264542 16590 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:43.267110 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:43.267750 16666 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:43.268621 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.271524 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.225010 21637 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:43.274852 21573 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:43.275830 21611 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:43.281708 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:43.285228 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.285895 16588 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:43.288858 21656 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.289338 21656 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:43.289690 21656 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.294819 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.295660 16596 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:43.296402 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.297327 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.298215 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.230533 21528 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:43.299160 21528 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.300136 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.300321 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.302212 21604 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.303570 21518 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.304001 21518 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:43.304379 21518 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.231392 21617 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.305171 21617 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.305904 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.307183 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.302819 21604 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.307723 21590 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:43.308324 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.309062 21604 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.309415 21617 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.310468 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.302242 16590 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:43.309975 21590 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:43.268167 21630 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.314720 20945 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.319962 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.319823 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:43.315003 16588 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:43.315944 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.313500 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.323799 16634 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.319737 20945 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.326119 21413 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:43.326412 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:43.327005 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:43.327672 20945 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:43.327858 16671 raft_consensus.cc:606] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.325309 21634 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:19:43.326608 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.327989 21633 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:43.328866 21633 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.329313 21633 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:43.326975 21526 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:19:43.311431 21650 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:19:43.324015 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.327703 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.336468 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:43.336706 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53262.8us
I20260709 13:19:43.338174 20945 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:43.341416 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.345810 16666 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:43.347539 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:43.344125 16634 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.351086 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.352624 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:43.357087 21590 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.358165 17413 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: []
I20260709 13:19:43.349287 21525 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.359908 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:43.360436 21525 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:43.360862 21525 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.362562 16593 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:43.364609 21640 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.365108 21640 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.351970 21473 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:19:43.365903 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 399869us
I20260709 13:19:43.367996 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.312765 21637 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:43.369879 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.373555 16561 leader_election.cc:400] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:43.373927 16561 leader_election.cc:403] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:43.374570 20945 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:43.375751 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.378232 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.378541 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.382716 17413 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:43.382808 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.383637 16666 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:43.383543 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.385303 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.379258 16662 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:19:43.345403 21634 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:43.394595 17414 raft_consensus.cc:2410] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:19:43.401077 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:43.401100 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.372676 21473 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:43.349718 21631 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.352509 21419 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:43.419410 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.421562 17411 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:43.367470 21392 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:43.422345 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.422451 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.359951 21350 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.374351 21608 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:43.424860 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.431397 17411 raft_consensus.cc:2410] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:19:43.350325 21518 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.381910 21655 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:19:43.418723 21640 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.421792 21473 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:43.366722 21633 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.451220 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.453980 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.432539 21650 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:43.432284 21605 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:43.457926 16738 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:43.459230 16740 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:43.368376 21611 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:19:43.464941 21633 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:43.463378 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64812.7us
I20260709 13:19:43.471200 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:43.472692 16666 raft_consensus.cc:606] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.473057 16666 raft_consensus.cc:614] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:43.475678 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.479194 21660 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:43.479696 21660 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:43.479883 21496 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.480547 21651 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:43.481648 21638 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:43.479971 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.481798 21672 raft_consensus.cc:993] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:19:43.483175 21672 raft_consensus.cc:1081] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:19:43.480424 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.479717 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.433502 21634 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:19:43.480122 21660 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:43.461630 21650 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:43.461369 21605 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:43.586344 17411 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:19:43.586808 17411 pending_rounds.cc:85] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 9
I20260709 13:19:43.587067 17411 pending_rounds.cc:107] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:19:43.587312 17411 raft_consensus.cc:2889] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:19:43.588908 21600 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:19:43.594512 21525 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.595997 16745 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:43.598137 21525 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.538554 21650 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:43.535936 21596 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.614055 16634 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.536876 21651 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:43.557615 21605 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:43.572456 21293 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.657954 21473 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.663347 21651 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.693866 21473 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.674110 21630 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.675025 21518 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.635401 21293 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.696751 21293 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:43.674135 21660 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:43.698549 21625 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:19:43.700451 21660 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.681885 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:43.682583 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.686939 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.680408 17411 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:19:43.710417 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.717551 20945 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:43.718053 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.634122 21638 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:43.722724 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.723330 16595 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:43.617699 21596 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.723352 16588 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:19:43.726626 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 297854us
I20260709 13:19:43.729506 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:43.730163 16738 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:43.743824 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:43.666441 21528 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.744796 16588 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:43.747071 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.747117 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.761616 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.754689 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:43.762538 16736 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:19:43.769593 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:43.773542 21639 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:19:43.720343 21660 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.719851 21293 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.775458 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.779855 16594 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:19:43.723694 21413 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:43.796729 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48834.6us
I20260709 13:19:43.764117 21604 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:43.798913 21214 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:19:43.805431 21612 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.807279 21662 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:43.801877 21524 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:43.817070 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:43.838778 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:43.824258 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:43.827832 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58337.2us
I20260709 13:19:43.840569 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.720590 21650 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.841588 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:43.843124 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:43.843884 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.845091 16671 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:43.846388 16666 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:19:43.826478 21392 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.853219 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:43.825651 21651 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:19:43.793807 21595 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:19:43.867018 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.797819 21518 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:19:43.870400 16669 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:19:43.834226 21644 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:43.818819 21645 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:19:43.829569 21214 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:43.873461 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.873721 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.877606 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:43.834178 21675 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.881198 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:43.865983 21592 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:43.847285 21604 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:43.891178 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.836133 21587 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:19:43.892225 16662 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:19:43.844956 21612 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.849412 21629 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:19:43.875411 21608 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:43.885031 21214 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:43.870792 21392 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:43.879768 21644 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.917198 17415 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 13:19:43.922379 21637 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.003s
I20260709 13:19:43.922941 21637 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.926292 21637 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:43.926707 16665 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:43.927659 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:43.932673 21392 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:43.933542 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:43.935302 21604 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.936117 21621 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:43.936497 21621 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:43.933934 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:43.945569 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:43.956918 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.911075 21599 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:43.966954 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:43.967695 16589 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:43.975752 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:43.976756 16669 raft_consensus.cc:2410] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:19:43.978097 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.980049 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:43.981393 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:43.984827 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:43.987260 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:19:43.987612 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 105907us
I20260709 13:19:43.982934 21559 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:43.990578 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:43.937229 21621 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:43.988588 21681 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 13:19:43.975646 21685 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.057s	user 0.003s	sys 0.005s
I20260709 13:19:43.992139 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:43.971092 21640 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:43.958938 16739 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:19:44.036772 16739 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:44.037328 16739 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:43.994565 17410 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:43.979512 21528 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:43.939505 21605 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:43.989302 21469 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:44.046937 16744 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:19:43.991628 21672 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:19:44.020059 21656 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.730s	user 0.008s	sys 0.006s
W20260709 13:19:44.055850 21690 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:19:44.036473 21640 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.059173 21599 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:44.064988 21596 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.066649 21612 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:44.086843 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54456.4us
I20260709 13:19:44.018455 21630 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:44.106523 21621 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:19:44.123154 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 132791us
W20260709 13:19:44.124202 21661 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:19:44.124881 21698 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.069s	user 0.003s	sys 0.000s
W20260709 13:19:44.122902 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 249527us
I20260709 13:19:44.135004 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:44.136754 16739 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.124086 21559 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:44.119138 21645 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:44.148366 21645 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.131217 21630 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.151160 21630 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:44.157289 21691 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:44.158038 21691 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:44.126765 21650 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.162245 21644 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.124809 21656 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.176680 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:44.177084 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.173847 16739 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:44.178133 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.177636 16671 raft_consensus.cc:2410] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:19:44.186764 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 51831.3us
I20260709 13:19:44.188213 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:44.188419 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.174685 16596 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:44.189177 16669 raft_consensus.cc:2376] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15. Re-sending same reply.
I20260709 13:19:44.179016 21588 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:44.126227 21640 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:44.194242 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:44.195312 21650 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:19:44.195726 21650 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:44.196122 21650 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:44.198228 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:44.186820 16670 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:44.198559 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.184036 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:44.200063 16671 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:19:44.201656 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.202685 16667 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:44.204106 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.206274 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.212087 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:44.214385 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:44.215206 16738 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:44.317375 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 139699us
W20260709 13:19:44.317410 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 109936us
I20260709 13:19:44.325219 16664 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:19:44.325860 16664 pending_rounds.cc:85] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 8
I20260709 13:19:44.326157 16664 pending_rounds.cc:107] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:19:44.326474 16664 raft_consensus.cc:2889] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:44.329035 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:44.330973 16744 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:44.344902 16667 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:19:44.400727 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:44.402995 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:44.406665 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 77735.2us
I20260709 13:19:44.409998 21518 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:44.411149 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:44.208386 21644 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:44.206154 21660 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:44.419466 16738 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:44.425580 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.333416 21662 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:44.194274 21214 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.207819 21588 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.215s
W20260709 13:19:44.419601 21622 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.959s	user 0.023s	sys 0.040s
I20260709 13:19:44.473197 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:44.473980 16664 raft_consensus.cc:606] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.474352 16664 raft_consensus.cc:614] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:44.478394 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.479226 16671 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:44.479511 16671 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:44.471158 21645 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.415269 21605 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:44.423259 21559 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:44.415164 21697 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:44.481594 21496 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.487758 16671 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:44.480866 21622 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.039s	user 0.024s	sys 0.040s
I20260709 13:19:44.487855 21697 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:44.450592 21660 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:44.527814 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 204885us
W20260709 13:19:44.532989 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60031.5us
I20260709 13:19:44.534250 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.488142 21645 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:19:44.542506 21621 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:44.543402 21611 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:44.490506 21559 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.571522 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.567405 21671 raft_consensus.cc:993] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:19:44.527179 21697 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:44.512995 21496 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:44.578413 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 370639us
W20260709 13:19:44.578271 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 156010us
I20260709 13:19:44.578804 17411 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:44.580564 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:44.587736 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.571899 21611 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:44.594727 21611 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.596710 21611 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:44.596875 21662 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:44.598278 21662 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:44.599840 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:44.622617 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:44.623744 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:44.623865 16590 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:44.625515 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.627199 16739 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:44.627745 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:44.629231 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.630048 16592 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:44.610924 21621 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:44.578083 21709 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:19:44.644522 21660 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:44.676906 16592 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:19:44.653028 21650 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:44.655236 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.603418 21214 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:44.612182 21496 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.601408 21559 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.615749 21630 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:44.605635 21573 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:44.590735 21608 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:19:44.662436 16670 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:44.601375 21350 raft_consensus.cc:1081] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:19:44.637815 21701 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:44.692337 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.695638 16669 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:44.696770 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.599895 21710 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.083s
I20260709 13:19:44.699112 21413 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.711560 16662 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:44.725966 21413 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:44.728349 21650 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:44.728911 21413 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.666539 21604 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:44.731426 16587 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:19:44.700524 21605 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:19:44.733249 21630 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:44.736375 21640 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.545s	user 0.005s	sys 0.004s
I20260709 13:19:44.614995 21662 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.739830 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 3480) took 1131 ms. Trace:
I20260709 13:19:44.740211 16596 rpcz_store.cc:276] 0709 13:19:43.608367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.608578 (+   211us) service_pool.cc:224] Handling call
0709 13:19:43.608971 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:44.189254 (+580283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.739596 (+550342us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400052154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:44.739773 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1300618496}
I20260709 13:19:44.738826 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 574) took 1294 ms. Trace:
I20260709 13:19:44.743389 16594 rpcz_store.cc:276] 0709 13:19:43.444192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.444350 (+   158us) service_pool.cc:224] Handling call
0709 13:19:43.468007 (+ 23657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.813917 (+345910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.738510 (+924593us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400190fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:44.738734 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716292352}
I20260709 13:19:44.673229 21518 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:19:44.747490 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 159865us
I20260709 13:19:44.736972 21640 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.750012 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:44.753595 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.748944 21697 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.769789 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:44.781299 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:44.782548 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.786048 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:19:44.790252 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 218860us
I20260709 13:19:44.794275 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.795207 16594 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:44.761467 21293 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:44.812223 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48962.4us
I20260709 13:19:44.817936 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:44.818799 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.820101 21608 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:44.821130 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:44.821730 16669 raft_consensus.cc:1813] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Handling vote request from an unknown peer 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:44.822372 16669 raft_consensus.cc:2410] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:19:44.823848 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:44.824700 16670 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:44.791585 21714 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:44.834741 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:44.835615 16588 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:44.835225 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 153193us
I20260709 13:19:44.839159 16665 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:44.815804 21630 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:44.851262 21650 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:44.865666 16588 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:44.853261 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.853569 21713 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:19:44.853968 21350 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:44.855057 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.852130 21697 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:44.871359 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52924.8us
I20260709 13:19:44.813467 21671 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:19:44.873283 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.890336 16595 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:44.822259 21622 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:19:44.894237 16591 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:19:44.894470 21715 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:44.894726 21650 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:44.897182 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.899346 21671 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:19:44.899236 21353 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:44.899964 21671 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:44.902377 16670 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
W20260709 13:19:44.905725 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51090.2us
I20260709 13:19:44.906328 21390 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:44.906615 16559 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.907337 21630 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:44.907768 21630 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:44.908655 21630 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.909943 21630 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:44.896986 21707 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:44.911621 21722 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:44.914651 21640 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.915758 21620 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:44.916203 21620 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:44.916527 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:44.917069 21620 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:44.900887 21650 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:44.911389 21707 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.919305 21650 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.920578 21650 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:44.920779 21707 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:44.922580 21722 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:44.938299 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:44.939200 21671 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.918318 21620 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:44.948545 21714 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:44.949218 21714 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:44.951188 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:44.951638 21714 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:44.952991 16739 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:19:44.953188 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 1712) took 1512 ms. Trace:
I20260709 13:19:44.953569 16739 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:44.953992 16739 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:44.954833 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:44.956499 21608 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:44.957015 21608 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:44.953908 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:44.959170 16745 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 7. Current term is 8. Ops: []
I20260709 13:19:44.953599 16742 rpcz_store.cc:276] 0709 13:19:43.440904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.441050 (+   146us) service_pool.cc:224] Handling call
0709 13:19:43.441335 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:19:43.447412 (+  6077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:19:43.447442 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:19:43.447912 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:19:44.723049 (+1275137us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b180022e100. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:19:44.816155 (+ 93106us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:19:44.816197 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:44.816260 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:19:44.952898 (+136638us) raft_consensus.cc:1685] finished
0709 13:19:44.952918 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:19:44.953130 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5725,"raft.queue_time_us":14102,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":8345,"spinlock_wait_cycles":7196544,"thread_start_us":92915,"threads_started":1,"wal-append.queue_time_us":1405877}
I20260709 13:19:44.965456 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:44.966747 21599 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:44.967378 21599 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.969456 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:44.970127 16670 raft_consensus.cc:606] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:44.970486 16670 raft_consensus.cc:614] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:44.973794 17414 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:44.974947 16739 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:44.977253 16743 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:44.982189 16593 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:44.988698 16742 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:44.989595 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:19:44.991410 16736 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:44.995764 16739 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:44.996757 16743 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:45.004292 16742 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:44.998642 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.019488 21599 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:45.005146 21608 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.023217 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.023957 16738 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:45.025751 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.027297 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.028554 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.030552 21644 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:45.032944 21469 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:19:45.035975 21353 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.037341 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1057) took 1178 ms. Trace:
I20260709 13:19:45.037396 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:45.038265 16737 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:45.039968 21447 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:45.040421 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:44.983594 21716 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:45.042016 17415 rpcz_store.cc:276] 0709 13:19:43.858649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.859798 (+  1149us) service_pool.cc:224] Handling call
0709 13:19:43.860237 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:19:43.919603 (+ 59366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:19:45.037113 (+1117510us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:19:45.037265 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120163456}
I20260709 13:19:45.065462 21691 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.044147 21528 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:45.059716 21703 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:45.068194 21703 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.068571 21703 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:45.069732 21599 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:19:45.071439 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.059988 21353 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.075196 21704 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:45.075719 21390 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.076292 21390 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.078704 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.041831 21645 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:19:45.082219 21645 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:45.082612 21645 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:45.078475 21390 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.084089 21656 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.083952 21645 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.078413 21716 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.076535 21353 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.085932 21645 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:45.087122 21716 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.088405 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.088969 16739 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:45.094484 20945 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:45.100824 20945 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:45.097819 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:45.102732 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62385.1us
I20260709 13:19:45.109655 21469 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:45.110203 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.116773 20945 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.119724 20945 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:45.121464 16588 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:19:45.128347 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:45.117195 21353 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:45.129951 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:45.131681 16670 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:19:45.131996 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.130210 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.133675 16587 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
W20260709 13:19:45.123618 21651 consensus_queue.cc:1175] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "efa00eb5e7404570958fdebb6cd077e8" responder_term: 13 status { last_received { term: 12 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:19:45.126190 21469 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:19:45.150079 21469 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:45.154949 17415 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:19:45.150561 21469 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:45.126591 21640 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:19:45.138302 21452 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.141999 21353 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:45.216003 16666 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:19:45.216454 16666 pending_rounds.cc:85] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 12
I20260709 13:19:45.216682 16666 pending_rounds.cc:107] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:19:45.216923 16666 raft_consensus.cc:2889] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:45.218856 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:45.255834 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:45.294570 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75855.4us
I20260709 13:19:45.301283 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:45.304280 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.305051 17415 raft_consensus.cc:1813] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Handling vote request from an unknown peer 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.305632 17415 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:45.209309 21608 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 13:19:45.222174 21630 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.309182 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.309691 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.296667 21573 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:45.212585 21452 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.324434 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.303400 21716 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:45.332597 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.338835 17414 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:19:45.339310 17414 pending_rounds.cc:85] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 10
I20260709 13:19:45.339532 17414 pending_rounds.cc:107] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:19:45.339771 17414 raft_consensus.cc:2889] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:45.339828 21452 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:45.305513 21469 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.347306 21469 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:45.353364 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:45.354539 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.323624 21630 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.353724 21591 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:45.354979 21525 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:45.359265 21528 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.359932 21447 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:45.362928 21725 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:45.349802 21737 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:45.351976 21637 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.331647 21716 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.370956 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.346658 21703 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.329571 21353 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.370308 21528 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:45.365667 21712 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.382486 21638 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.376754 21573 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.387513 21612 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:45.388152 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.401103 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.403659 21736 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:19:45.404639 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.390784 21631 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.380715 21637 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.409369 21736 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 13:19:45.426770 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.399598 21638 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.432999 21638 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.402606 21573 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.439352 21630 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.440585 21716 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.442648 21703 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.407460 21413 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:19:45.425637 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:45.451911 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.452754 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.455793 16740 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:45.414076 21715 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.396389 21712 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.457525 21638 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.471231 21638 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.473366 21638 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.475975 21573 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.463971 21715 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.478900 21528 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.479326 21715 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.424535 21631 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.479848 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.480988 21528 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:45.481269 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.481546 21631 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.482105 16745 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:45.425350 21447 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:19:45.483445 21447 consensus_queue.cc:1243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:19:45.483978 21447 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:45.484221 21413 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:45.484674 21413 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:45.471231 21637 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.485617 21413 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.486511 21691 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:45.484727 21712 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:45.502082 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.504108 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.504783 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.505944 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.506937 16744 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:45.457968 21525 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:45.509572 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:45.516469 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.492161 21413 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:19:45.521704 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 149547us
I20260709 13:19:45.508426 21469 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:45.530726 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 105182us
I20260709 13:19:45.508595 21637 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:19:45.537424 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.535938 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.523744 21353 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:45.539223 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.540035 17415 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 11: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:45.542830 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:19:45.543926 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.548571 21525 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.534034 21736 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
W20260709 13:19:45.543152 21741 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:19:45.539768 21353 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.567413 21353 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:45.570942 21697 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:45.573428 21591 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.574052 21591 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.575101 16743 raft_consensus.cc:1813] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Handling vote request from an unknown peer 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:45.575717 16743 raft_consensus.cc:2410] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:19:45.577898 21591 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.579309 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.580022 17415 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:19:45.581468 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:45.582459 21691 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:45.583575 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.583878 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.584568 16592 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:45.585104 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.585848 17415 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:45.563388 21682 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:19:45.530678 21469 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.586720 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.593369 16593 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:45.593802 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.593984 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.584268 16588 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:19:45.597836 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.599148 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.603932 21736 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:45.605104 21749 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:45.605607 21749 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:19:45.607257 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.606822 21749 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.608135 21749 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:19:45.611368 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.627275 21676 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:45.627789 21676 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:45.628717 21676 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.629667 21671 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:19:45.630069 21676 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:45.632694 16671 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:45.635300 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.638737 21469 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.640201 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:45.641489 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 43879us
I20260709 13:19:45.644481 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.645314 21716 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:45.645839 21596 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.670020 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.643596 21637 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:45.670672 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:45.672300 16671 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:19:45.582967 21691 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:45.679126 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:45.687482 16634 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.692284 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.693198 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 2606) took 1214 ms. Trace:
I20260709 13:19:45.671388 21703 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:45.696391 21703 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.698765 21703 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:45.699411 21712 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:45.685437 21691 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:45.694191 16671 rpcz_store.cc:276] 0709 13:19:44.467183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:44.478112 (+ 10929us) service_pool.cc:224] Handling call
0709 13:19:44.864130 (+386018us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b18002ca1e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fc1f646 00007f327fb5958e 00007f327fbdaf9e 00007f327fbdc7de 00007f327fbe827c 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c
0709 13:19:44.921172 (+ 57042us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b58002633d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe827c 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c
0709 13:19:45.632346 (+711174us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b58002b19d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe827c 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c
0709 13:19:45.681257 (+ 48911us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2323508992}
I20260709 13:19:45.714428 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.676623 21640 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.719683 21573 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:45.685729 21596 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.794081 16488 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:45.735044 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.690918 21469 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:45.801564 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:45.679714 21637 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.713940 21633 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:45.848747 21633 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.849284 21633 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:45.796008 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:45.861639 21633 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.796080 21630 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:45.866658 21630 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.708093 21638 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:19:45.868898 21630 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.869297 21633 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.869843 17414 raft_consensus.cc:2410] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:45.869747 21630 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:45.794679 21596 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:45.769490 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53229.5us
I20260709 13:19:45.813138 21469 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.795771 16488 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 12, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 12 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:45.868950 21638 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:19:45.880481 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 236108us
I20260709 13:19:45.711921 21631 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:45.883737 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.900170 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.902993 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:45.892437 16634 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:45.812296 21704 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:45.808805 21573 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.716468 21715 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:45.905962 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:45.907908 16671 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:19:45.800750 21353 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.806728 21672 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:45.808450 21711 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:45.964232 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58546.7us
I20260709 13:19:45.678790 21716 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.887001 21637 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:45.945324 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:45.974787 16666 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:45.975185 21711 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.975430 21715 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.981756 21672 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:45.983484 21704 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:46.010170 21353 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:45.956684 21712 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:46.010812 21704 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.011798 21711 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.012102 21715 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:45.924047 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.018167 16740 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:19:45.920183 16742 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:45.902906 21638 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:46.034412 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:45.800158 21640 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.978698 21644 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:45.984843 21691 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.014026 21672 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.041293 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.022135 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.892498 21559 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.057967 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:46.058593 16740 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:46.058473 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.059235 16667 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.065625 16745 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:19:46.067415 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.069272 16561 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.070279 21651 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.070838 21651 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.073297 21651 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.074739 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:46.084535 21691 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.087714 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:46.088449 17411 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.097294 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:19:46.101341 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.105089 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.106542 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:45.901259 21469 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:45.901491 21630 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.115926 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.118335 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.119900 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.121275 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.111696 21636 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:46.127991 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:46.151608 21636 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.152112 21636 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:46.139771 21711 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.152796 21711 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.142598 21712 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:46.142946 16488 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:46.113276 21651 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:46.148815 16490 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 221) took 22957 ms (client timeout 15000 ms). Trace:
I20260709 13:19:46.138134 16740 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:46.168277 17411 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
W20260709 13:19:46.206846 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50542.2us
I20260709 13:19:46.212405 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.213478 16664 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:46.209110 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:46.009682 21573 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.125087 21611 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:46.150422 21637 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.227535 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:46.237442 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.241572 16664 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:46.094316 21559 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.207412 21390 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.155511 21691 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:46.207824 21385 raft_consensus.cc:2764] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:19:46.219614 21528 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:46.201606 16490 rpcz_store.cc:269] 0709 13:19:23.191542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:23.195046 (+  3504us) service_pool.cc:224] Handling call
0709 13:19:46.148715 (+22953669us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:42.727745 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:42.758068 (+ 30323us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:42.758131 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:19:42.758158 (+    27us) tablet.cc:662] Decoding operations
0709 13:19:42.764166 (+  6008us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:42.764277 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:42.764313 (+    36us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:19:42.766054 (+  1741us) tablet.cc:701] Row locks acquired
0709 13:19:42.766113 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:19:42.766320 (+   207us) write_op.cc:270] Start()
0709 13:19:42.766502 (+   182us) write_op.cc:276] Timestamp: P: 1783603182766254 usec, L: 0
0709 13:19:42.766544 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:19:42.848935 (+ 82391us) log.cc:844] Serialized 6638 byte log entry
0709 13:19:43.254270 (+405335us) op_driver.cc:464] REPLICATION: finished
0709 13:19:43.273649 (+ 19379us) write_op.cc:301] APPLY: starting
0709 13:19:43.273791 (+   142us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:43.657899 (+384108us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:43.658018 (+   119us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:43.688276 (+ 30258us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:43.688672 (+   396us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 13:19:43.688735 (+    63us) write_op.cc:312] APPLY: finished
0709 13:19:43.689204 (+   469us) log.cc:844] Serialized 156 byte log entry
0709 13:19:43.704911 (+ 15707us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:43.705111 (+   200us) write_op.cc:454] Released schema lock
0709 13:19:43.705195 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:43.724952 (+ 19757us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:43.724998 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27808,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":19018,"apply.run_cpu_time_us":30560,"apply.run_wall_time_us":443044,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75255,"num_ops":17,"prepare.queue_time_us":168072,"prepare.run_cpu_time_us":22077,"prepare.run_wall_time_us":253441,"raft.queue_time_us":208504,"raft.run_cpu_time_us":8857,"raft.run_wall_time_us":62692,"replication_time_us":467056,"thread_start_us":298338,"threads_started":4,"wal-append.queue_time_us":121303,"wal-append.run_cpu_time_us":38279,"wal-append.run_wall_time_us":1716996}]]}
W20260709 13:19:46.240620 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:46.252197 21711 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.237259 21592 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:46.259027 21390 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.259891 21691 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.260464 21637 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.263403 16664 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:46.291038 21592 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:46.299650 21591 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.300107 17414 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:46.300041 21592 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.300273 21591 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:46.112939 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44030.8us
I20260709 13:19:46.243587 21611 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.247815 21638 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.321240 21559 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:46.306738 16480 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81229us
I20260709 13:19:46.324432 21651 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:46.326421 21559 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:46.328032 21390 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.328450 21559 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:46.327654 21753 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:19:46.330842 21640 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.330925 21559 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.332973 21691 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:46.306607 21588 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:19:46.269351 21636 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.334213 21588 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:19:46.309464 16667 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:46.340122 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.346697 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.270681 21528 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.348677 16664 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: [15.8-17.9]
I20260709 13:19:46.352134 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.353870 16664 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:46.355199 21596 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.338078 21588 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:46.305456 21618 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:19:46.335623 21611 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.357369 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.369304 16667 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:46.333781 21716 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.335414 21573 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.377916 21591 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:46.337299 21636 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:46.375411 21691 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:46.422487 21691 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:46.432824 21691 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.434360 21691 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.435086 21630 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.436648 21638 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.384480 21716 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.368191 21596 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.439527 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.448251 16664 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:19:46.430802 16667 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:19:46.441331 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.394495 21469 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.394937 21611 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.415300 21704 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.336486 16488 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:46.462623 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.450764 21591 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:19:46.456115 16634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.039s	sys 0.095s
W20260709 13:19:46.479758 16634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.039s	sys 0.095s
I20260709 13:19:46.451771 21716 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.484021 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:46.482686 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 590339us
I20260709 13:19:46.460335 21611 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.438259 21588 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.489735 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.490442 16589 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:46.491068 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.491843 16665 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:46.463896 21704 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.478996 21622 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:46.496415 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.497805 16595 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:46.500465 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.501209 16589 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:46.483253 21707 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.478412 21591 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.526433 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.527901 16593 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 15 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:19:46.535516 21469 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.536386 21469 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.539160 21638 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.539680 21638 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.546540 21630 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.547145 21630 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.552304 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.554136 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.557395 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.505204 21622 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.562619 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.503949 21762 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.577442 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.578598 16665 raft_consensus.cc:2368] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:19:46.578328 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.578629 16596 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:46.583688 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.586217 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.588763 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.509142 21588 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:19:46.585317 16596 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:19:46.595808 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.598215 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.598857 16594 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:46.601491 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.602576 16590 raft_consensus.cc:2410] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 13:19:46.562602 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 461584us
I20260709 13:19:46.619349 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.620836 21390 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:46.621418 21390 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.580693 21559 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:46.624145 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.624442 16593 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
W20260709 13:19:46.635013 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 316621us
I20260709 13:19:46.508230 21707 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.643273 16664 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:46.648458 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.650547 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.650537 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.651621 16634 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.634960 21528 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.632941 21559 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:46.591796 21762 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:46.662887 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.666416 16665 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:19:46.675521 21766 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:19:46.687856 21716 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.647966 21469 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.658383 16636 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.656708 21711 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:46.669203 21644 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:46.694262 16664 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:46.660483 21712 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:46.654294 21630 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.702898 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.504317 16488 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 12 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:46.659435 21761 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:19:46.668737 21559 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:46.704641 16591 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:19:46.712288 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.717216 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.721608 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.699124 21644 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.726490 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:46.727402 16740 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:46.701941 21469 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:46.697973 21711 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.724948 21630 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:19:46.713125 21596 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.716513 21761 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.732722 16664 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:46.738140 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.724080 21631 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:46.740998 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.723737 21638 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.751281 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.752041 16742 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:46.746625 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.723419 21390 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.754974 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:46.755821 17415 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:46.757408 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.749754 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.730932 21644 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.759361 21622 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:46.759992 21591 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:46.760862 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.762064 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.736044 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.747836 21761 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:46.745810 21704 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.734170 21711 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.738921 21611 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.776124 17411 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:46.783104 16488 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 6 to 9, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:46.785079 16488 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 14, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:46.787508 21762 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:46.780781 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 141421us
W20260709 13:19:46.791867 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 140363us
I20260709 13:19:46.794466 21644 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.800004 21644 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.795959 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.804906 21636 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:46.808878 21636 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.812006 21612 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:46.812635 21612 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.815857 17410 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:46.816190 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.816758 16664 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.817744 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.818503 16663 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:46.812192 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.819234 21638 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.819774 21638 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.820091 21622 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.820439 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.820588 21622 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.821759 21638 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:46.822536 21707 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.823124 21716 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:46.823563 21716 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:46.823920 21716 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.824329 21559 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.825646 21385 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:46.827679 16669 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:19:46.826067 21772 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.166s	user 0.000s	sys 0.002s
I20260709 13:19:46.819271 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.828649 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.829504 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.829932 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.833884 16666 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:46.834457 21636 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.834847 21612 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.837725 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.839216 21596 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:46.840725 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:46.745457 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.841897 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.840873 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.842760 17415 raft_consensus.cc:2410] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:19:46.844060 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.844071 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.845407 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.839653 21596 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:19:46.845620 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49754.1us
I20260709 13:19:46.853196 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:46.853637 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.853850 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.854674 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:46.855541 17411 raft_consensus.cc:2410] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:19:46.857944 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.858525 16591 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:19:46.858589 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.859189 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.859401 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:46.859663 21782 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:46.859954 16745 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.860174 21782 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:46.822558 21622 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:46.861367 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:46.861085 21782 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:46.862085 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.846062 16664 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:46.848933 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.849937 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.865044 16634 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.865553 16668 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:19:46.866101 16708 leader_election.cc:400] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:19:46.851521 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.866887 16663 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:46.859198 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.845999 16742 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:46.849610 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:46.867617 16744 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:46.867755 16666 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:46.856029 16561 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:46.874894 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:46.852504 21587 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:19:46.854787 21596 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:46.887059 21587 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.887485 21587 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.854537 21469 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:46.893030 21761 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.894374 16665 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:46.879696 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.854609 21781 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:19:46.896500 21781 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.896835 21781 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:46.873180 21710 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:46.877547 21712 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:46.897985 21712 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.898350 21712 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:46.900575 21559 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.880388 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:46.902895 21469 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:46.903332 16738 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:46.852469 21611 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:19:46.903863 21469 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.904778 21651 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:46.786814 16488 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 15, leader changed from <none> to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 15 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:46.908560 16488 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:46.909991 21611 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.910975 21761 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:46.910694 21611 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:46.924315 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50152.5us
I20260709 13:19:46.883950 21704 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:46.916368 16591 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:19:46.918022 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.927806 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:46.918834 21591 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:19:46.924659 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49871.8us
I20260709 13:19:46.925112 16745 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:19:46.928063 16593 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:46.929450 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:46.918936 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:46.931263 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 58007.7us
I20260709 13:19:46.939182 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:19:46.939853 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:46.942997 16593 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:46.944721 16634 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.946219 21758 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:46.948654 21773 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:46.949124 21773 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.949528 21773 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:46.949281 21716 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:46.952919 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96985us
I20260709 13:19:46.954228 16591 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:46.956727 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.957649 16594 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:46.951714 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.959434 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:46.950958 21591 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:46.962524 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:46.926424 21630 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:46.930047 21672 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:46.966365 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.967054 16595 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:46.948063 21704 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:46.952255 21758 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:46.963841 21761 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:46.965552 21644 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:46.938300 21637 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:19:46.969271 21630 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:46.910346 16488 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 12, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 12 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:46.969772 21651 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:19:46.973450 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:46.972203 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:46.997402 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:46.998713 16591 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:19:46.982609 21637 consensus_queue.cc:1243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:19:47.006959 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44376.9us
I20260709 13:19:47.011443 16595 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:47.010520 16593 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:19:46.984267 21651 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:46.990976 21644 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:47.018720 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.020717 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:47.022962 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 63990.5us
I20260709 13:19:47.011297 21672 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:47.015934 21630 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.015314 21716 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.020143 21638 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:47.021350 21651 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:47.032124 21596 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.039330 16488 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 13 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:47.042232 21611 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.011974 21559 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:19:47.042891 21710 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:47.049182 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:47.065001 16667 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:47.020750 21612 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.027688 21762 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:47.062773 21596 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.064472 16591 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:19:47.069151 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.070086 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:47.026322 21704 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.057967 21758 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.021099 21591 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.080497 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.058636 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.074317 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.073714 16559 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:47.081957 16595 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:47.070458 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:47.079349 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.082748 21596 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.082937 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:47.085621 21611 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:47.087391 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.089787 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.091637 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.092260 16745 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:19:47.093245 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:47.092285 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.093290 21749 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.097509 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.072997 16488 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:47.098052 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:47.079928 21612 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:47.090337 21596 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.099466 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:47.115550 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.117740 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:47.118189 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.080758 21644 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.082923 21789 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.098573 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.120811 16737 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:47.094638 21762 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:47.088860 21630 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:47.122520 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:47.082245 21712 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.120415 21789 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.123651 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:47.125123 21672 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.118853 21596 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.106988 21711 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.129601 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.132257 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.132555 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.133607 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.094169 21622 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:19:47.137280 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.126467 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.140802 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.097183 21611 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.168604 16736 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:47.114172 21749 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:47.248634 16740 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:47.249948 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.249212 16738 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
W20260709 13:19:47.254756 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.119836 21587 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.140038 21789 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:47.265522 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:47.267845 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:47.271122 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:47.279253 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.280412 21587 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.281040 16589 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:47.280848 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.388636 16592 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:47.168627 21390 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:19:47.283444 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.126518 21782 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:19:47.347600 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.348798 17415 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:19:47.427045 17415 pending_rounds.cc:85] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 10
I20260709 13:19:47.427346 17415 pending_rounds.cc:107] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:19:47.427608 17415 raft_consensus.cc:2889] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:47.348552 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:47.142810 16488 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 7 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 12 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:47.356060 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.351842 21447 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:47.352351 21771 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:19:47.369877 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.138599 21672 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:47.373008 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:19:47.379235 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 238502us
I20260709 13:19:47.335099 21773 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.386510 16669 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:19:47.447008 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74075.1us
I20260709 13:19:47.338929 21781 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:47.457701 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 186430us
I20260709 13:19:47.477660 16738 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:47.438699 21782 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:47.487535 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.488634 16593 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:47.452250 21672 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.429143 21390 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:47.476760 21762 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:47.498615 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.499620 16738 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:19:47.498709 21712 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:47.503244 21793 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:19:47.493950 21353 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:19:47.507419 21795 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:19:47.497325 21762 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:47.510097 16738 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:19:47.510710 21762 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.511148 21651 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:47.513638 21792 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:19:47.514228 16596 raft_consensus.cc:2368] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:19:47.516047 21651 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.516813 21390 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:47.527170 21611 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.530731 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:47.541458 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55375.9us
I20260709 13:19:47.542793 21704 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:47.544001 21716 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:47.547236 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:47.548854 16593 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:19:47.549625 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 62179us
I20260709 13:19:47.550473 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.552177 21781 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.538003 21638 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:47.553968 21638 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:47.554427 21773 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.555003 21638 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.555382 21782 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.557855 21789 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.559193 16745 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:47.540603 21762 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.561512 21762 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.564811 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.565766 16737 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.576072 21596 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.578073 16587 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:47.578388 21596 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.588172 21736 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:47.589157 21736 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:47.594611 21800 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:47.596259 21804 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:19:47.600378 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:47.616222 16737 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:47.624312 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:47.676316 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 76016.2us
I20260709 13:19:47.677335 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:47.678830 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:47.699477 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:19:47.704909 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:19:47.705542 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.007s	sys 0.009s
W20260709 13:19:47.706805 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 82630.2us
W20260709 13:19:47.706147 21809 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:19:47.705839 16737 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:47.569176 21762 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 13:19:47.712415 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:47.712740 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:47.713070 16587 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:47.713492 16588 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:47.714694 16593 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:47.719360 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.718675 21736 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:47.720000 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.769714 16739 raft_consensus.cc:2368] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
W20260709 13:19:47.705401 21710 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.000s	sys 0.016s
I20260709 13:19:47.752880 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.752285 16740 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:47.744390 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.705222 21781 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:47.697706 21596 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:47.705120 21804 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:47.754369 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.732450 21637 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:47.792164 21637 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:47.743939 21761 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:19:47.723395 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.795572 16742 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:47.722517 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:47.706149 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.007s	sys 0.009s
W20260709 13:19:47.797391 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 705839us
I20260709 13:19:47.706477 21800 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:47.717730 21469 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:47.708621 21672 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.732070 21797 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:47.731797 21758 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:47.816902 21758 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.816802 21797 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:47.789994 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45692.2us
I20260709 13:19:47.757651 16587 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
W20260709 13:19:47.788467 21710 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.000s	sys 0.020s
I20260709 13:19:47.801693 16667 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:19:47.805178 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83510.6us
I20260709 13:19:47.820158 16663 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:47.793905 21588 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:47.847595 16670 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:47.853406 16587 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:47.850804 16742 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:19:47.821379 21789 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:47.821636 21469 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:47.933856 21469 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.935309 21637 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:47.892892 21630 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:47.937251 21630 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:47.939881 21630 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.894186 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.896466 21638 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:47.860105 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:47.867838 21758 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:47.942559 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 82550.9us
I20260709 13:19:47.943738 21787 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:19:47.944787 21749 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:47.900673 21818 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:19:47.901365 21819 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:19:47.907387 17410 pending_rounds.cc:85] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 9
I20260709 13:19:47.909636 21612 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:47.966178 17410 pending_rounds.cc:107] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:19:47.966550 17410 raft_consensus.cc:2889] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:47.911211 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:47.911494 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:47.912428 21825 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:19:47.912808 17419 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:19:47.919224 21672 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:47.920565 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:47.971740 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51274us
I20260709 13:19:47.972898 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:47.973400 16708 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:47.922194 21797 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:47.926523 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:19:47.976310 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 159460us
I20260709 13:19:47.979264 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.979442 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:47.980444 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:47.981247 16590 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:19:47.981528 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:47.929267 21816 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:19:47.930073 21682 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:47.963955 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:47.965070 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.000164 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:48.002444 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:48.000478 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.034507 16634 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:48.036264 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 71304.2us
I20260709 13:19:48.038010 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.039400 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:48.040196 16667 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:19:47.966468 21810 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:48.043067 21761 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:19:48.034246 21618 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:19:48.049440 21781 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.032758 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.029202 21824 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.048489 16667 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
W20260709 13:19:48.052325 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51929.9us
I20260709 13:19:48.000968 21736 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:19:48.059446 17419 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:48.059538 21353 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:48.060393 17413 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:19:48.059597 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:48.061563 17414 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:48.007509 21691 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.007069 21757 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:48.057219 21447 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:48.064437 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:48.006234 21682 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:48.006438 21749 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:48.006930 21672 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:48.007299 21588 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:19:48.027700 21811 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:19:48.037228 21797 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:19:48.062702 21761 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:48.068701 21447 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:48.067456 21824 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.105600 21711 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:48.105875 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:48.114638 16663 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.050082 21810 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:48.121778 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 146602us
I20260709 13:19:48.122823 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:19:48.146405 16480 catalog_manager.cc:5220] ChangeConfig:REMOVE_PEER RPC for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to CAS failure; no further retry: Illegal state: Request specified cas_config_opid_index of 6 but the committed config has opid_index of 9
I20260709 13:19:48.113238 21447 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:48.147905 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:48.072502 21691 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.087925 21672 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.093307 21749 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.180380 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.148084 21797 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:19:48.209831 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:48.217550 21618 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:19:48.216423 21591 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:48.179237 21469 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:48.225564 16663 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:48.227008 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:48.228066 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.221532 21591 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.244293 21834 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:19:48.245625 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 215796us
I20260709 13:19:48.258498 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.259521 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.248311 21749 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:48.223233 21797 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:48.257213 21591 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.229251 21469 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.276968 21711 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.262542 21834 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.279516 21834 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.280825 21682 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.282356 21682 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:48.297678 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 245095us
I20260709 13:19:48.299947 21810 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:48.300674 21588 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:19:48.301661 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 191265us
I20260709 13:19:48.301836 21691 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:48.298571 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.300251 21797 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.303812 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.306694 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.301232 21588 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:48.309008 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:48.309166 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 82279.3us
I20260709 13:19:48.311007 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.295563 21672 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:48.332592 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.337553 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.339151 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:48.340171 16662 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:19:48.309741 21390 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.380793 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.334079 21789 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.381963 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.384478 21827 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:19:48.386718 16663 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
W20260709 13:19:48.387593 21840 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:19:48.388315 21832 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:19:48.388760 21469 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:48.389137 21828 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:19:48.391017 16670 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:48.393419 21447 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:48.396035 21838 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:19:48.396438 21588 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.335616 21781 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:19:48.473615 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.341472 21528 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 13:19:48.474642 16744 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:19:48.335484 21691 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:19:48.476742 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.421344 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:48.414083 16665 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:48.424680 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.424999 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.431242 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.435880 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.436884 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.445066 16737 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:48.445946 16740 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:48.406791 21805 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.483290 21447 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:48.489481 21691 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:48.489956 21691 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:48.491163 21588 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.491367 16588 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:19:48.412024 21799 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.427397 21827 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:48.498796 21827 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.500133 21827 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:19:48.501391 21528 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:48.501844 21528 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:48.502668 21528 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.503929 21528 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:48.420450 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.490854 21691 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.493816 16591 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:19:48.408406 21824 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:48.478601 16668 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:48.497453 21799 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.413045 21591 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:48.512041 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 66887.2us
W20260709 13:19:48.516575 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 179077us
I20260709 13:19:48.518975 16561 leader_election.cc:400] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:48.519598 16561 leader_election.cc:403] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.524114 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.508744 21761 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:48.525334 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.528488 16636 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.524422 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:48.532832 21800 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:19:48.533106 21761 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:48.533457 21800 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:48.533685 21761 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.522284 21651 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:48.535037 21811 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:19:48.535714 21799 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:48.504477 21762 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:19:48.534375 21800 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.537685 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:48.539129 21833 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:48.540038 21833 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.540465 21833 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.532985 21789 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:48.543033 16742 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:48.543852 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.545300 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.548456 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.525802 21691 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:48.539112 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:48.532914 21469 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.556710 21848 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:48.567586 21848 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.567953 21848 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:48.549921 21638 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.553082 21762 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:19:48.575542 16741 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:19:48.578183 21845 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:48.580325 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.205s (SENT) [suppressed 2 similar messages]
I20260709 13:19:48.581357 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.582221 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.582710 21588 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:48.581115 21596 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:19:48.583357 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.592417 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:48.593389 16596 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:48.593428 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.596589 16594 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:19:48.596861 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 260790us
I20260709 13:19:48.594951 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.600906 21588 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:48.591220 21596 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:48.601969 21591 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:48.605037 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.604694 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.589857 21469 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:48.608582 21588 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:48.609091 21588 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:48.609993 21588 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:48.614240 21835 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.310s	user 0.000s	sys 0.004s
I20260709 13:19:48.614789 21588 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.615242 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.616017 21390 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:19:48.617278 16559 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.579007 21845 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.619181 16591 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:19:48.627077 16591 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:48.631093 16669 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:48.583333 21762 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:19:48.632603 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:48.614259 21651 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.625142 21787 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:48.636336 21787 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:48.636413 21804 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:48.637019 21804 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.637269 21787 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.639523 21804 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:48.638643 21834 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.644353 16588 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:48.645547 16741 raft_consensus.cc:2368] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14.
W20260709 13:19:48.647223 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 198111us
I20260709 13:19:48.650290 21672 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:19:48.650441 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 57328.4us
I20260709 13:19:48.613785 21833 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.651353 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.652455 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.657696 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.658373 16666 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:19:48.658722 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:48.659287 17410 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:48.660266 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 79962.8us
I20260709 13:19:48.663487 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.665680 16741 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:48.667287 17376 leader_election.cc:400] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14.
I20260709 13:19:48.667656 17376 leader_election.cc:403] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.668493 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.667176 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.655364 21644 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:19:48.669927 21630 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:48.670799 21630 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:48.671165 21630 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:48.672009 21630 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.673079 21630 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.673556 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:48.687245 16594 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:19:48.688916 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.669551 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.700661 21644 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:48.673907 21385 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:19:48.698828 21591 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:48.704821 21848 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.705901 16742 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:48.659646 21834 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:48.651342 21469 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:19:48.697904 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:48.674294 21757 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:48.723707 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:48.726480 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:48.730072 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:48.729120 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:48.732491 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:48.692974 21808 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:48.667497 21651 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:19:48.673707 21672 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:19:48.747344 16588 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:19:48.748451 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:48.698804 21636 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:19:48.732900 21804 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:48.706346 21761 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.809715 21808 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:48.806280 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73887us
I20260709 13:19:48.750662 21644 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:19:48.838678 21804 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:48.856143 16669 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:48.716742 21559 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:19:48.838856 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88350.6us
I20260709 13:19:48.771016 21848 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:48.840253 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:48.867504 16662 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:48.838912 21800 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:48.843978 21622 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:48.875586 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:48.887082 16736 raft_consensus.cc:2368] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:48.889070 16561 leader_election.cc:400] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:48.889495 16561 leader_election.cc:403] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:19:48.844128 21846 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:48.861524 21804 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:48.875126 21622 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:48.896029 21855 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:19:48.869939 21559 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:48.922596 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:49.005626 21845 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:49.007102 21528 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.004s
I20260709 13:19:49.009071 21559 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:48.871243 21800 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.025177 17410 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
W20260709 13:19:49.039750 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:49.053944 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 205575us
W20260709 13:19:49.013640 21853 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
W20260709 13:19:49.054191 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 160593us
I20260709 13:19:49.103295 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:49.051828 21823 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14.
I20260709 13:19:49.124156 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:49.124804 17410 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.122900 21592 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:49.129992 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:49.105419 21761 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:49.135406 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65198.2us
I20260709 13:19:49.109241 21833 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:49.142220 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.147188 17410 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:49.150916 21592 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:19:49.180470 21827 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.002s
I20260709 13:19:49.182081 21622 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:19:49.208951 21868 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:19:49.216603 16736 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:49.221557 21592 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:49.222636 21800 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:49.214402 21630 consensus_meta.cc:287] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.528s	user 0.004s	sys 0.007s
I20260709 13:19:49.240446 16665 raft_consensus.cc:2410] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 13:19:49.255244 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 94820.4us
I20260709 13:19:49.256278 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:49.257931 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:49.258980 21757 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:49.259704 21757 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:19:49.261896 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:49.262024 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:49.262662 17414 raft_consensus.cc:2368] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:19:49.269507 21804 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.273130 21855 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
W20260709 13:19:49.275527 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 95986.4us
I20260709 13:19:49.292829 21773 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:49.293310 21773 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:49.293844 16668 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:19:49.300917 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:49.302099 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:49.294128 21773 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.308394 21758 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.308939 21758 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.310992 21630 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.311484 21630 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.313635 21630 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:49.314109 21630 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.314836 21353 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:19:49.315582 21867 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:19:49.315402 21353 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:49.317570 21353 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:49.319330 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:49.319881 16668 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:49.326005 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.326532 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.286440 21869 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:19:49.315434 21758 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:49.336390 21711 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.337062 21711 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:49.337553 21876 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.180s	user 0.000s	sys 0.002s
I20260709 13:19:49.339890 21869 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:49.340354 21869 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:49.341800 21869 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:49.343083 21869 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:49.346827 21860 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 13:19:49.314574 21630 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:49.352620 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:49.354359 16561 leader_election.cc:400] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
W20260709 13:19:49.314599 21864 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.183s	user 0.003s	sys 0.000s
I20260709 13:19:49.360268 21871 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:49.339965 21711 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:49.375029 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:49.381381 21862 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.274s	user 0.003s	sys 0.000s
I20260709 13:19:49.384730 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:49.387111 21866 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.336s	user 0.003s	sys 0.000s
I20260709 13:19:49.387225 21469 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.387635 21630 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:49.387821 21469 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.389935 21469 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:49.393047 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:49.393776 16665 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.400108 21773 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:49.411595 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:49.414741 16744 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:49.419909 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:49.421454 16668 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:19:49.422170 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:49.424448 21804 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:19:49.426919 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 123934us
I20260709 13:19:49.428673 21856 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:19:49.431725 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:49.432751 16670 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:49.438076 16488 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 144) took 6909 ms. Trace:
I20260709 13:19:49.401736 16708 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:49.451467 16594 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
W20260709 13:19:49.407716 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 45712.8us
I20260709 13:19:49.443266 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:49.454555 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52896.3us
I20260709 13:19:49.476326 21592 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:49.499469 16594 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:19:49.503038 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52093.1us
I20260709 13:19:49.529661 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:49.530972 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:19:49.504395 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45399.3us
I20260709 13:19:49.452411 16488 rpcz_store.cc:276] 0709 13:19:42.645320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.658380 (+ 13060us) service_pool.cc:224] Handling call
0709 13:19:42.843194 (+184814us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:19:43.090084 (+246890us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:19:49.437975 (+6347891us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:47.786366 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:47.912962 (+126596us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:47.913000 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:19:47.913018 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:47.914575 (+  1557us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:47.914672 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:47.914700 (+    28us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:19:47.915884 (+  1184us) tablet.cc:701] Row locks acquired
0709 13:19:47.915920 (+    36us) write_op.cc:260] PREPARE: finished
0709 13:19:47.916083 (+   163us) write_op.cc:270] Start()
0709 13:19:47.965069 (+ 48986us) write_op.cc:276] Timestamp: P: 1783603187916031 usec, L: 0
0709 13:19:47.967213 (+  2144us) op_driver.cc:348] REPLICATION: starting
0709 13:19:48.108166 (+140953us) log.cc:844] Serialized 5173 byte log entry
0709 13:19:48.670380 (+562214us) op_driver.cc:464] REPLICATION: finished
0709 13:19:48.812690 (+142310us) write_op.cc:301] APPLY: starting
0709 13:19:48.812790 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:49.232835 (+420045us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:49.232939 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:49.378384 (+145445us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:49.378667 (+   283us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 13:19:49.378715 (+    48us) write_op.cc:312] APPLY: finished
0709 13:19:49.379074 (+   359us) log.cc:844] Serialized 121 byte log entry
0709 13:19:49.379277 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:49.379445 (+   168us) write_op.cc:454] Released schema lock
0709 13:19:49.379500 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:49.412157 (+ 32657us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:49.412191 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":562206,"reported_tablets":21,"spinlock_wait_cycles":298627456,"child_traces":[["op",{"apply.queue_time_us":79087,"apply.run_cpu_time_us":22508,"apply.run_wall_time_us":602325,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58519,"num_ops":13,"prepare.queue_time_us":222565,"prepare.run_cpu_time_us":70478,"prepare.run_wall_time_us":567175,"raft.queue_time_us":110787,"raft.run_cpu_time_us":4556,"raft.run_wall_time_us":25789,"replication_time_us":700883,"thread_start_us":378193,"threads_started":4,"wal-append.queue_time_us":352170}]]}
I20260709 13:19:49.533427 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:49.534734 21710 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:49.514192 16491 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.535782 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:49.536624 21528 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:49.538486 21528 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:49.539671 21528 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.401296 16665 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:49.468178 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:49.526590 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.522535 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.527304 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.543725 16592 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:49.543852 16587 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:49.547784 16593 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:49.550541 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:49.551414 16665 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:49.554822 16670 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:19:49.542696 16491 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 10, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:49.561563 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 154748us
I20260709 13:19:49.562713 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:19:49.564541 16670 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:19:49.559286 16491 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 17, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.588851 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:49.589900 16736 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:49.592330 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:49.588205 16491 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 17. New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.596540 16491 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 17. New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.592931 16736 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:19:49.567986 21672 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:49.568287 21630 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:19:49.610059 16665 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:19:49.612481 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:49.618458 16668 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:19:49.623752 16491 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 10, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 10 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.625873 16491 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.628883 16491 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.631597 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:49.634934 16491 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 13 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.637768 16491 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.637997 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:49.642442 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:49.639984 16491 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.645009 16491 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.647224 16491 catalog_manager.cc:5697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.649735 16491 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 2 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.659991 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:49.664579 21871 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:49.665337 21871 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:49.666430 21871 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.670027 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:49.671368 21353 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:49.672150 21353 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:49.672667 21353 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:49.669526 21870 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:19:49.679170 21528 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:49.680204 16491 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 12, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.669981 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:49.682972 16491 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 12, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.684502 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:49.685263 16590 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:49.684774 16491 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.695045 21528 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:49.696132 21528 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:49.706645 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:49.710623 17410 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:49.710943 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:49.712685 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:49.715508 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:49.717869 21833 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:19:49.718281 21762 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:49.713680 16491 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.714188 21758 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:49.716830 21644 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:19:49.723729 21833 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:49.724591 21591 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:19:49.732770 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:49.706478 21870 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:19:49.734586 21469 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:19:49.736717 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:49.718798 21762 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.714363 21631 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:49.749889 16491 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 10 to 12, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 12 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.751781 21591 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:49.744357 21884 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:19:49.772511 21762 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:49.776003 16491 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.780045 16491 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 11 to 12, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.782351 16491 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:49.784909 16491 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.776439 21863 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.787111 16491 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 5 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:49.772320 21469 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:19:49.796309 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:49.777058 21711 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:49.801849 17411 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:49.796881 16491 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 10 to 11, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 11 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:49.786408 21353 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:49.795783 21528 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:49.776733 21789 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:19:49.796010 21469 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:49.791079 21863 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:49.819659 21863 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.804939 21711 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:49.839182 21711 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:49.863063 16590 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:49.865499 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46302.3us
W20260709 13:19:49.866992 21893 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:19:49.873235 21591 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:49.885174 16590 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:49.817533 21528 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:49.977543 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 202445us
W20260709 13:19:49.980357 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 95414.6us
I20260709 13:19:50.002967 21833 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
W20260709 13:19:49.991020 21887 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.105s	user 0.000s	sys 0.005s
I20260709 13:19:50.020339 21863 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.046953 16668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:50.047765 16668 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:50.048166 16668 raft_consensus.cc:614] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:50.053295 21591 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:19:50.085881 21469 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.088374 21469 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.089790 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:50.090847 16590 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:19:50.095352 21899 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:19:50.112784 21863 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:50.116294 21528 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:50.128054 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:50.128736 16670 raft_consensus.cc:606] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:50.129062 16670 raft_consensus.cc:614] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:50.028894 21810 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.132941 21810 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:50.135212 21810 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:50.146041 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:50.147612 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:50.148228 16594 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:50.159658 16591 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:19:50.160079 16591 pending_rounds.cc:85] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 9
I20260709 13:19:50.160349 16591 pending_rounds.cc:107] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:19:50.160677 16591 raft_consensus.cc:2889] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.157734 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:50.174247 16593 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:50.126319 21762 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.158067 21804 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:19:50.193652 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:50.195356 16669 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: [15.8-18.10]
I20260709 13:19:50.210384 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:50.210843 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:50.211256 16669 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:19:50.211534 16665 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:19:50.219480 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:50.249065 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 65647.4us
I20260709 13:19:50.200273 21804 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:50.253738 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55800.6us
I20260709 13:19:50.242287 21827 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:19:50.266105 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:50.266909 16665 raft_consensus.cc:606] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:50.262811 21827 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.271870 21827 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:50.279608 21804 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.288689 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:50.291074 21872 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:50.292477 21872 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:50.292980 21872 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:50.293833 21872 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.303426 16740 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:19:50.302619 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:50.304018 16740 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 9
I20260709 13:19:50.304389 16736 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:50.304382 16740 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:19:50.305100 16740 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:50.307631 16591 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:50.309635 17410 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:50.265829 21711 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:50.311918 16670 raft_consensus.cc:2410] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:19:50.320320 16744 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:19:50.332878 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:50.333557 16670 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:50.333642 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:50.334301 16665 raft_consensus.cc:606] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:50.334656 16665 raft_consensus.cc:614] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:50.335965 16670 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:19:50.342701 21638 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.343544 21638 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.344648 16665 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:50.350713 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:50.351696 21761 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:19:50.354327 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:50.346436 21638 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.356195 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:50.357139 16591 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:50.357862 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:50.359555 21883 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:50.360160 21883 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:19:50.360258 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54809.2us
I20260709 13:19:50.360677 21883 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:19:50.359599 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62269.3us
I20260709 13:19:50.368455 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:50.352408 21761 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:50.372355 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:50.376997 16591 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:19:50.381924 21811 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:50.399334 21762 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.403473 21761 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.421360 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:50.427819 21711 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:50.377733 21638 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:50.430198 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:50.431015 17410 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:19:50.434530 16740 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:19:50.435983 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:50.429471 21638 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:50.445746 21638 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:50.461654 21827 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.464303 21638 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.465718 21901 raft_consensus.cc:993] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:50.466359 21901 raft_consensus.cc:1081] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:50.467168 21638 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.468210 21883 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.471292 21827 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:50.471333 21883 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.501978 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:19:50.509001 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 150577us
I20260709 13:19:50.516043 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:50.515671 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 156612us
I20260709 13:19:50.518754 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:50.520750 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:50.522755 16587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:50.523857 16587 raft_consensus.cc:606] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Received request to transfer leadership
W20260709 13:19:50.533028 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96994.8us
W20260709 13:19:50.543610 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 96843.8us
I20260709 13:19:50.547730 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:50.548676 21711 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:50.549654 21711 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:50.551774 21385 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:50.553071 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:50.554282 16588 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:50.559681 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:50.559244 21711 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.563580 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:50.564646 16593 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:50.565363 16593 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:50.616940 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:50.547780 21651 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:19:50.624553 21773 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:50.636076 16665 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:19:50.637347 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:50.637447 16593 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.638259 16669 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:50.646001 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 103319us
I20260709 13:19:50.662139 16593 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:50.673516 16591 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:50.679127 16588 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
W20260709 13:19:50.677389 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 150969us
I20260709 13:19:50.681213 16662 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:50.689296 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:50.703473 16669 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:50.703927 21863 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:50.731150 16740 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:19:50.729936 21849 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:50.756910 21921 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:50.757848 21921 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:50.775369 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:50.775666 21919 tablet_replica.cc:333] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:19:50.776298 16587 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:50.790472 21849 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:50.793308 21849 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:50.793375 21921 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:19:50.806679 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53506.8us
I20260709 13:19:50.808128 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:50.808822 16742 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:50.812110 16587 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:19:50.823590 16588 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:50.824276 16588 raft_consensus.cc:606] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Received request to transfer leadership
I20260709 13:19:50.829967 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:50.852504 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:50.845661 21833 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:19:50.873085 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:50.873852 16591 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
W20260709 13:19:50.876714 21925 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:19:50.883314 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 72028.5us
I20260709 13:19:50.886330 16663 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:19:50.887877 21924 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:19:50.884634 21869 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:50.897397 21869 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:50.898538 21869 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.916280 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:50.917222 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65773.8us
I20260709 13:19:50.918484 21622 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:50.919101 21622 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:50.920389 21622 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:50.931666 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:19:50.933318 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 186992us
I20260709 13:19:50.934576 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:50.936043 21901 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:50.924885 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:50.963009 21810 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:19:50.976400 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:50.977226 16666 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:19:50.977576 16666 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:50.979535 16742 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:50.967973 21901 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:50.978576 16666 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:50.986899 21928 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:19:50.973073 21762 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:50.997321 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:50.998279 21762 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:50.999854 21923 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.187s	user 0.001s	sys 0.004s
I20260709 13:19:50.998303 16664 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:19:50.999804 21762 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.001866 21901 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:51.009547 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 75116.9us
I20260709 13:19:51.011725 16666 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:19:51.040139 21870 raft_consensus.cc:993] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
W20260709 13:19:51.067047 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 238776us
I20260709 13:19:51.040988 21870 raft_consensus.cc:1081] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
W20260709 13:19:51.089740 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76762.8us
W20260709 13:19:51.112845 21930 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:19:51.110327 16666 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:19:51.130522 16595 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
I20260709 13:19:51.131913 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:51.133813 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:51.135576 16588 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:51.167220 16740 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:19:51.160955 21869 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:19:51.191521 21871 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:19:51.192847 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:19:51.193890 17411 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:51.208840 16665 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:19:51.212286 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63654us
I20260709 13:19:51.179782 21869 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:19:51.198509 21871 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:19:51.242527 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 67053.5us
I20260709 13:19:51.222141 21869 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.248855 16662 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36074
I20260709 13:19:51.249694 16662 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:51.252030 16662 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.238003 21638 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
W20260709 13:19:51.267956 21931 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:19:51.267562 21871 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:51.269124 21846 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.269791 21846 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:51.307921 16662 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.310019 16666 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
W20260709 13:19:51.327981 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 182621us
I20260709 13:19:51.330021 16662 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:51.327211 21846 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:51.331130 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:51.332243 21933 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
W20260709 13:19:51.334473 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.334266 21559 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:19:51.339607 21937 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:19:51.349172 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 182097us
I20260709 13:19:51.362671 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:51.364833 16587 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:51.365425 16587 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:51.379317 21799 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:51.380153 21799 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.367985 21919 raft_consensus.cc:2243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:51.390811 16738 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:51.390799 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:51.392042 16739 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.397986 21934 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:19:51.404700 21869 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.435663 16488 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 17, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 17 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:51.438074 21863 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:51.437717 16488 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:51.438597 21863 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:51.440349 21848 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:19:51.441134 21848 consensus_queue.cc:1243] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:19:51.442013 21944 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:51.442381 21944 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:51.443372 21944 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:51.443107 16587 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.490962 21919 raft_consensus.cc:2272] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Raft consensus is shut down!
W20260709 13:19:51.494335 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 73995.6us
W20260709 13:19:51.495504 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 66304.2us
I20260709 13:19:51.495908 16587 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.498114 16739 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:51.498889 16591 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:19:51.501089 21944 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.510459 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:51.525328 16593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42550
W20260709 13:19:51.502640 21870 raft_consensus.cc:670] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped [suppressed 3 similar messages]
I20260709 13:19:51.507642 21863 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:51.542408 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:51.543009 16669 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.551424 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:51.557898 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:51.560405 21941 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:19:51.572417 16594 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:19:51.575897 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:51.576856 16666 raft_consensus.cc:2410] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:19:51.578094 16665 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:19:51.579267 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51823.6us
I20260709 13:19:51.586655 21919 ts_tablet_manager.cc:1916] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:51.591038 21869 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:51.592835 16669 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:51.603097 21799 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:51.594184 21952 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:19:51.615273 21952 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:19:51.615727 21952 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.623091 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:51.624008 16588 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:19:51.628212 21940 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:19:51.629045 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:51.631949 16588 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:51.632949 21651 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:51.633522 21651 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:51.634155 21940 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:51.634490 21651 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.597412 16587 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:51.606621 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:19:51.593554 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83220.9us
I20260709 13:19:51.603299 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:19:51.652833 16593 raft_consensus.cc:606] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Received request to transfer leadership
I20260709 13:19:51.654107 16593 raft_consensus.cc:614] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:51.650133 16488 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 16, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 16 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:51.671736 16738 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:19:51.672210 16738 pending_rounds.cc:85] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 8
I20260709 13:19:51.672576 16738 pending_rounds.cc:107] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:19:51.672964 16738 raft_consensus.cc:2889] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:19:51.671310 16488 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:51.675763 16745 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:19:51.687736 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:19:51.690027 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:19:51.675817 16488 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 12, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:51.708284 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:51.710362 16670 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:51.721613 21804 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:51.733882 16488 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:51.740170 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:51.749933 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:51.750679 17414 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:51.757995 16592 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:19:51.759392 21939 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:51.759647 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:51.760180 21946 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:19:51.764534 17414 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:19:51.779207 21934 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:51.781548 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:19:51.786092 21939 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:19:51.888592 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97740.3us
I20260709 13:19:51.892823 16488 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:51.895740 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 151374us
I20260709 13:19:51.896564 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:51.735574 21952 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:51.907315 16708 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:51.787411 21939 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:51.942245 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 53911.9us
I20260709 13:19:51.943053 17412 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:51.925338 16488 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 5 to 11, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 11 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:51.948129 21870 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:19:51.948792 17410 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:51.950431 21952 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:51.950740 21919 ts_tablet_manager.cc:1929] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 10.8
I20260709 13:19:51.951406 21919 log.cc:1199] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/4951b0fadf5b43e49a71f0bb4fb7a85e
I20260709 13:19:51.952826 21919 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42550 (request call id 1) took 1434 ms. Trace:
I20260709 13:19:51.953238 21919 rpcz_store.cc:276] 0709 13:19:50.518818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:50.520308 (+  1490us) service_pool.cc:224] Handling call
0709 13:19:50.751275 (+230967us) ts_tablet_manager.cc:1151] Deleting tablet 4951b0fadf5b43e49a71f0bb4fb7a85e
0709 13:19:50.762633 (+ 11358us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:19:51.702791 (+940158us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:51.950576 (+247785us) tablet_metadata.cc:665] Metadata flushed
0709 13:19:51.952727 (+  2151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":645105,"tablet-delete.queue_time_us":208644,"thread_start_us":64423,"threads_started":1}
I20260709 13:19:51.955713 21863 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:19:51.956199 21962 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 13:19:51.948264 21920 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:19:51.934762 21799 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:19:51.964167 17412 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
W20260709 13:19:52.020850 21958 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:19:52.047236 16745 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:19:52.052101 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 273780us
I20260709 13:19:51.970033 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:52.063787 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:52.064543 16669 raft_consensus.cc:2368] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
W20260709 13:19:52.064916 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 157728us
W20260709 13:19:52.066040 16478 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 96151.4us
I20260709 13:19:52.070935 21944 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:19:52.074713 16668 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:19:52.075794 21870 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:52.079126 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:52.088315 21918 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:52.094162 21941 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:52.096742 17374 leader_election.cc:400] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:19:52.097081 17374 leader_election.cc:403] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:19:52.098465 21849 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
W20260709 13:19:52.117972 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49142.7us
I20260709 13:19:52.210284 17410 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:19:52.263334 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81830.5us
I20260709 13:19:52.306461 16491 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 237) took 9467 ms. Trace:
I20260709 13:19:52.341658 16665 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:19:52.419785 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 179785us
I20260709 13:19:52.428469 21871 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:19:52.458252 21871 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:52.461258 21871 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:52.356784 21990 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
W20260709 13:19:52.512799 21991 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:19:52.514664 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 82418.4us
I20260709 13:19:52.347411 16491 rpcz_store.cc:276] 0709 13:19:42.839171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:42.840591 (+  1420us) service_pool.cc:224] Handling call
0709 13:19:43.056244 (+215653us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:19:52.306224 (+9249980us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:50.253293 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:50.260318 (+  7025us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:50.260372 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:19:50.260397 (+    25us) tablet.cc:662] Decoding operations
0709 13:19:50.298932 (+ 38535us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:50.299019 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:50.299044 (+    25us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:19:50.311506 (+ 12462us) tablet.cc:701] Row locks acquired
0709 13:19:50.311525 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:19:50.311662 (+   137us) write_op.cc:270] Start()
0709 13:19:50.311773 (+   111us) write_op.cc:276] Timestamp: P: 1783603190311616 usec, L: 0
0709 13:19:50.311799 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:19:50.312534 (+   735us) log.cc:844] Serialized 9690 byte log entry
0709 13:19:50.469482 (+156948us) op_driver.cc:464] REPLICATION: finished
0709 13:19:50.476079 (+  6597us) write_op.cc:301] APPLY: starting
0709 13:19:50.476166 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:50.920659 (+444493us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:50.920847 (+   188us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:51.187337 (+266490us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:51.187837 (+   500us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 13:19:51.187894 (+    57us) write_op.cc:312] APPLY: finished
0709 13:19:51.188682 (+   788us) log.cc:844] Serialized 220 byte log entry
0709 13:19:51.190615 (+  1933us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:51.191426 (+   811us) write_op.cc:454] Released schema lock
0709 13:19:51.191545 (+   119us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:51.247581 (+ 56036us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:51.248951 (+  1370us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5556,"reported_tablets":50,"spinlock_wait_cycles":35400704,"child_traces":[["op",{"apply.queue_time_us":6252,"apply.run_cpu_time_us":44585,"apply.run_wall_time_us":739400,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108727,"num_ops":25,"prepare.queue_time_us":207989,"prepare.run_cpu_time_us":7499,"prepare.run_wall_time_us":73015,"replication_time_us":143263,"thread_start_us":172945,"threads_started":2}]]}
I20260709 13:19:52.612404 16742 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:52.616060 21901 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:52.623940 17410 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:52.656972 21972 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
W20260709 13:19:52.721083 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 298825us
I20260709 13:19:52.723954 16593 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:19:52.738907 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:19:52.744683 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:42550
W20260709 13:19:52.768133 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 341103us
W20260709 13:19:52.852540 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 66722.5us
W20260709 13:19:52.859632 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 67516.5us
I20260709 13:19:52.860736 21918 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:19:52.863759 16542 auto_rebalancer.cc:893] Removing replace marker failed: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:52.864251 16542 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:19:52.864548 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:19:52.784993 21951 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:19:52.880621 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:19:52.892747 22005 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:19:52.932441 22012 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.115s	user 0.004s	sys 0.000s
W20260709 13:19:52.960546 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52686.5us
I20260709 13:19:52.976159 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:52.981688 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:52.992048 21651 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:19:53.107825 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69228.5us
I20260709 13:19:53.111753 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:19:53.112366 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 74188.5us
I20260709 13:19:53.422654 16591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:19:53.423666 16591 raft_consensus.cc:606] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:53.424072 16591 raft_consensus.cc:614] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:53.442487 16591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:19:53.443133 16591 raft_consensus.cc:606] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:53.443532 16591 raft_consensus.cc:614] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:53.447467 16590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:19:53.448174 16590 raft_consensus.cc:606] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:19:53.448519 16590 raft_consensus.cc:614] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:53.500592 16488 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 145) took 3865 ms. Trace:
I20260709 13:19:53.511482 16488 rpcz_store.cc:276] 0709 13:19:49.641116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.654348 (+ 13232us) service_pool.cc:224] Handling call
0709 13:19:53.498973 (+3844625us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:52.357782 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:52.371044 (+ 13262us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:52.371121 (+    77us) write_op.cc:435] Acquired schema lock
0709 13:19:52.371172 (+    51us) tablet.cc:662] Decoding operations
0709 13:19:52.559515 (+188343us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:52.571008 (+ 11493us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:52.580481 (+  9473us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:19:52.627849 (+ 47368us) tablet.cc:701] Row locks acquired
0709 13:19:52.627922 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:19:52.628098 (+   176us) write_op.cc:270] Start()
0709 13:19:52.628254 (+   156us) write_op.cc:276] Timestamp: P: 1783603192628039 usec, L: 0
0709 13:19:52.628278 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:19:52.629031 (+   753us) log.cc:844] Serialized 3170 byte log entry
0709 13:19:53.147877 (+518846us) op_driver.cc:464] REPLICATION: finished
0709 13:19:53.324091 (+176214us) write_op.cc:301] APPLY: starting
0709 13:19:53.325024 (+   933us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:53.403159 (+ 78135us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:53.403255 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:53.416097 (+ 12842us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:53.416329 (+   232us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:19:53.416376 (+    47us) write_op.cc:312] APPLY: finished
0709 13:19:53.416666 (+   290us) log.cc:844] Serialized 81 byte log entry
0709 13:19:53.419153 (+  2487us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:53.419276 (+   123us) write_op.cc:454] Released schema lock
0709 13:19:53.419328 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:53.424689 (+  5361us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:53.424723 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":212,"mutex_wait_us":313670,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":84883,"apply.run_cpu_time_us":12345,"apply.run_wall_time_us":186367,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":168114,"prepare.run_cpu_time_us":30413,"prepare.run_wall_time_us":395254,"raft.queue_time_us":227733,"raft.run_cpu_time_us":22247,"raft.run_wall_time_us":75165,"replication_time_us":505332,"thread_start_us":364615,"threads_started":4,"wal-append.queue_time_us":134641}]]}
I20260709 13:19:53.555157 17414 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:53.601953 16741 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:53.610785 21854 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:53.623868 16590 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:19:53.567389 16744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:19:53.544721 16494 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:53.625089 16744 raft_consensus.cc:606] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:53.627368 21854 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:19:53.702991 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62596.1us
I20260709 13:19:53.708483 16591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:19:53.709169 16591 raft_consensus.cc:606] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:53.725083 16595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:19:53.727321 16595 raft_consensus.cc:606] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:53.727648 16595 raft_consensus.cc:614] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:19:53.738672 21854 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Committing config change with OpId 15.13: config changed from index 8 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.781831 16741 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 8 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.805243 21950 raft_consensus.cc:993] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:53.820844 21950 raft_consensus.cc:1081] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:53.816977 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:19:53.823019 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:19:53.824472 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64719.3us
I20260709 13:19:53.840677 16590 raft_consensus.cc:2955] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 8 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:53.848187 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 70435.9us
I20260709 13:19:53.845413 16667 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36074
I20260709 13:19:53.848888 16667 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:53.849272 16667 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.864377 16667 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:53.868657 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:53.869328 16595 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:19:53.869638 16595 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:53.870326 16595 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:53.871431 16595 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.878312 16667 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:53.884428 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:53.885044 16744 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:19:53.886688 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:19:53.890547 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52236.9us
I20260709 13:19:53.930298 16595 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:19:53.936218 16488 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.941462 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:53.943562 16744 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:19:53.947620 16488 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 14, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.953769 16488 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 13 to 14, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.956801 16488 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.948046 21976 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:19:53.958384 16488 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 14, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:53.964663 21976 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:54.007611 21976 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:54.008491 22035 raft_consensus.cc:993] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:54.009102 21848 raft_consensus.cc:1081] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:54.021068 16666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36082
I20260709 13:19:54.021723 16666 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:54.022123 16666 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:54.048444 16666 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:54.078655 16666 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:54.084323 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:54.085043 16594 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:19:54.096820 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:54.097450 16744 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:19:54.097721 16744 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:54.110262 16744 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:54.112011 16744 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:19:54.117903 16494 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 214) took 44506 ms (client timeout 15000 ms). Trace:
I20260709 13:19:54.120878 16744 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
W20260709 13:19:54.118429 16494 rpcz_store.cc:269] 0709 13:19:09.611716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:09.612167 (+   451us) service_pool.cc:224] Handling call
0709 13:19:54.117782 (+44505615us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:53.719154 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:53.737561 (+ 18407us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:53.737600 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:19:53.737616 (+    16us) tablet.cc:662] Decoding operations
0709 13:19:53.737963 (+   347us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:53.738011 (+    48us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:53.738047 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:19:53.738228 (+   181us) tablet.cc:701] Row locks acquired
0709 13:19:53.738245 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:19:53.738384 (+   139us) write_op.cc:270] Start()
0709 13:19:53.738498 (+   114us) write_op.cc:276] Timestamp: P: 1783603193738334 usec, L: 0
0709 13:19:53.738517 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:19:53.748971 (+ 10454us) log.cc:844] Serialized 535 byte log entry
0709 13:19:53.920457 (+171486us) op_driver.cc:464] REPLICATION: finished
0709 13:19:53.920895 (+   438us) write_op.cc:301] APPLY: starting
0709 13:19:53.920981 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:53.921825 (+   844us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:53.921932 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:53.922748 (+   816us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:53.922962 (+   214us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:19:53.923010 (+    48us) write_op.cc:312] APPLY: finished
0709 13:19:53.923211 (+   201us) log.cc:844] Serialized 25 byte log entry
0709 13:19:53.923370 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:53.923460 (+    90us) write_op.cc:454] Released schema lock
0709 13:19:53.923498 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:53.923909 (+   411us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:53.923937 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3433,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":2970,"apply.run_wall_time_us":2970,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":86183,"prepare.run_cpu_time_us":3674,"prepare.run_wall_time_us":32689,"replication_time_us":164206,"thread_start_us":58795,"threads_started":1}]]}
I20260709 13:19:54.134196 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:54.135684 21972 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:19:54.136226 21972 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:54.137156 21972 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:54.144481 16594 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:19:54.159929 16488 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 8 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 15 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:19:54.216130 16490 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 17 to 18, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:54.222324 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:59566
I20260709 13:19:54.223418 16490 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 5 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:54.228901 16490 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 6 to 17, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:54.241938 16490 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 15 to 17, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
W20260709 13:19:54.248852 16542 auto_rebalancer.cc:629] Failed to schedule move for tablet e8b0d179ad714b59b34ab1dc57a57c02: Illegal state: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
W20260709 13:19:54.255507 16542 auto_rebalancer.cc:629] Failed to schedule move for tablet 5bfdfe1aa047409c9214ba1f43eb434a: Illegal state: Replica 170921ba08b74a33995ae7c7686f8109 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:54.250214 16490 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 10 to 12, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:54.263130 16666 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:54.276005 16490 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 11 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:54.279318 16490 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 14 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:54.282903 22042 tablet_replica.cc:333] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:19:54.284688 22042 raft_consensus.cc:2243] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:54.285813 22042 raft_consensus.cc:2272] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:54.284198 16490 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:54.293313 16490 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:54.295334 16490 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 10 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:54.297302 16490 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:54.298759 16490 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 10 to 11, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 11 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:54.310847 16595 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:19:54.311904 16738 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:19:54.311089 16636 consensus_peers.cc:597] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e0b4dc6fa23d44edb449d1be0842b184. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:54.313055 21972 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:54.315140 21976 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:19:54.322576 22042 ts_tablet_manager.cc:1916] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:54.382809 22040 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:54.387032 16594 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:54.433203 16744 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Committing config change with OpId 12.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:54.458722 22042 ts_tablet_manager.cc:1929] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.12
I20260709 13:19:54.459360 22042 log.cc:1199] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/fc0ed8d0a60640a0a5ce3a772c3eef5c
I20260709 13:19:54.478083 16666 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:54.461987 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet fc0ed8d0a60640a0a5ce3a772c3eef5c (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
W20260709 13:19:54.498911 16636 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1a71b94ecfb4155a5cfe591077e9a1c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:54.501942 16744 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:54.502444 16595 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:54.505908 21972 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:54.507339 21976 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:54.511322 16738 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:54.517144 21976 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:54.590732 21972 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:54.598436 16738 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:54.618788 16594 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:54.620721 16595 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:19:54.622779 21976 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:54.610680 16744 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:54.631114 21951 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:54.670300 16490 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 216) took 4937 ms. Trace:
I20260709 13:19:54.671556 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:59566
I20260709 13:19:54.686931 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet fc0ed8d0a60640a0a5ce3a772c3eef5c (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:54.670822 16490 rpcz_store.cc:276] 0709 13:19:49.732628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:49.732845 (+   217us) service_pool.cc:224] Handling call
0709 13:19:54.670205 (+4937360us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:54.305947 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:54.306595 (+   648us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:54.306642 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:19:54.306664 (+    22us) tablet.cc:662] Decoding operations
0709 13:19:54.307987 (+  1323us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:54.308049 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:54.308085 (+    36us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:19:54.309241 (+  1156us) tablet.cc:701] Row locks acquired
0709 13:19:54.309264 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:19:54.309436 (+   172us) write_op.cc:270] Start()
0709 13:19:54.309562 (+   126us) write_op.cc:276] Timestamp: P: 1783603194309381 usec, L: 0
0709 13:19:54.309587 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:19:54.313283 (+  3696us) log.cc:844] Serialized 4738 byte log entry
0709 13:19:54.316121 (+  2838us) op_driver.cc:464] REPLICATION: finished
0709 13:19:54.316606 (+   485us) write_op.cc:301] APPLY: starting
0709 13:19:54.316694 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:54.326096 (+  9402us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:54.326211 (+   115us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:54.381359 (+ 55148us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:54.381666 (+   307us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:19:54.381724 (+    58us) write_op.cc:312] APPLY: finished
0709 13:19:54.382096 (+   372us) log.cc:844] Serialized 113 byte log entry
0709 13:19:54.395327 (+ 13231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:54.395479 (+   152us) write_op.cc:454] Released schema lock
0709 13:19:54.395527 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:54.419401 (+ 23874us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:54.419443 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26641,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":274,"apply.run_cpu_time_us":16380,"apply.run_wall_time_us":102714,"cfile_cache_hit":48,"cfile_cache_hit_bytes":50208,"num_ops":12,"prepare.queue_time_us":143,"prepare.run_cpu_time_us":4819,"prepare.run_wall_time_us":21101,"replication_time_us":6379}]]}
I20260709 13:19:54.782848 16594 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:19:54.784713 22040 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:19:54.824023 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:54.852398 22055 ts_tablet_manager.cc:933] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:54.863219 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:42550
W20260709 13:19:54.864094 16492 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 236) took 29813 ms (client timeout 15000 ms). Trace:
W20260709 13:19:54.867002 16492 rpcz_store.cc:269] 0709 13:19:25.051017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:25.086148 (+ 35131us) service_pool.cc:224] Handling call
0709 13:19:54.863852 (+29777704us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133206,"reported_tablets":50}
I20260709 13:19:54.906131 22055 tablet_copy_client.cc:323] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:19:54.896735 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:54.923580 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet e0b4dc6fa23d44edb449d1be0842b184 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:19:54.924216 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet e0b4dc6fa23d44edb449d1be0842b184 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-e0b4dc6fa23d44edb449d1be0842b184
I20260709 13:19:54.937038 16681 tablet_copy_source_session.cc:215] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:54.950206 22055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0b4dc6fa23d44edb449d1be0842b184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:19:55.013751 16489 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.027176 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 4951b0fadf5b43e49a71f0bb4fb7a85e with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 9)) from {username='slave'} at 127.0.0.1:42550
W20260709 13:19:55.036732 16493 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 235) took 50418 ms (client timeout 15000 ms). Trace:
W20260709 13:19:55.037242 16493 rpcz_store.cc:269] 0709 13:19:04.618086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:04.623734 (+  5648us) service_pool.cc:224] Handling call
0709 13:19:55.036526 (+50412792us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29600,"reported_tablets":50}
I20260709 13:19:55.040076 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 4951b0fadf5b43e49a71f0bb4fb7a85e (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:55.051998 22061 ts_tablet_manager.cc:933] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:19:55.080613 22061 tablet_copy_client.cc:323] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:19:55.081074 22055 tablet_copy_client.cc:806] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:55.081840 22055 tablet_copy_client.cc:670] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:19:55.089214 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet f1a71b94ecfb4155a5cfe591077e9a1c from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:19:55.089876 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet f1a71b94ecfb4155a5cfe591077e9a1c from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-f1a71b94ecfb4155a5cfe591077e9a1c
I20260709 13:19:55.096653 22055 tablet_copy_client.cc:538] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:55.105134 16681 tablet_copy_source_session.cc:215] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:19:55.109401 22061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1a71b94ecfb4155a5cfe591077e9a1c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:19:55.208073 16489 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 222) took 11725 ms (client timeout 15000 ms). Trace:
I20260709 13:19:55.216024 22055 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:19:55.222395 22061 tablet_copy_client.cc:806] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:19:55.223263 22061 tablet_copy_client.cc:670] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
W20260709 13:19:55.208595 16489 rpcz_store.cc:269] 0709 13:19:43.482626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:43.482934 (+   308us) service_pool.cc:224] Handling call
0709 13:19:44.739397 (+1256463us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:19:55.207995 (+10468598us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:55.053128 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:55.053777 (+   649us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:55.053813 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:19:55.053831 (+    18us) tablet.cc:662] Decoding operations
0709 13:19:55.054171 (+   340us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:55.054244 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:55.054270 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:19:55.054475 (+   205us) tablet.cc:701] Row locks acquired
0709 13:19:55.054493 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:19:55.054645 (+   152us) write_op.cc:270] Start()
0709 13:19:55.054753 (+   108us) write_op.cc:276] Timestamp: P: 1783603195054586 usec, L: 0
0709 13:19:55.054773 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:19:55.070922 (+ 16149us) log.cc:844] Serialized 535 byte log entry
0709 13:19:55.080553 (+  9631us) op_driver.cc:464] REPLICATION: finished
0709 13:19:55.128543 (+ 47990us) write_op.cc:301] APPLY: starting
0709 13:19:55.128642 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:55.129865 (+  1223us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:55.129986 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:55.139275 (+  9289us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:55.139516 (+   241us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:19:55.139586 (+    70us) write_op.cc:312] APPLY: finished
0709 13:19:55.139834 (+   248us) log.cc:844] Serialized 25 byte log entry
0709 13:19:55.144083 (+  4249us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:55.144204 (+   121us) write_op.cc:454] Released schema lock
0709 13:19:55.144258 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:55.145224 (+   966us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:55.145264 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":77124,"reported_tablets":51,"spinlock_wait_cycles":35328384,"child_traces":[["op",{"apply.queue_time_us":47660,"apply.run_cpu_time_us":3900,"apply.run_wall_time_us":16652,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"num_ops":1,"prepare.queue_time_us":12037,"prepare.run_cpu_time_us":2773,"prepare.run_wall_time_us":18149,"replication_time_us":25613,"thread_start_us":42448,"threads_started":2}]]}
I20260709 13:19:55.246244 22061 tablet_copy_client.cc:538] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:19:55.288187 22061 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:19:55.366648 16491 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.368690 16491 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 18 to 19, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 19 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:55.371374 16491 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:55.396703 16491 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 7 to 9, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.400823 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet fc0ed8d0a60640a0a5ce3a772c3eef5c with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 8 (current committed config index is 13)) from {username='slave'} at 127.0.0.1:59566
I20260709 13:19:55.405937 16491 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.408414 16491 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.409952 16491 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:55.423893 16491 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:55.425621 16491 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 16 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.427222 16666 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:55.420357 16487 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 215) took 26933 ms (client timeout 15000 ms). Trace:
W20260709 13:19:55.429625 16487 rpcz_store.cc:269] 0709 13:19:28.486934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:28.487174 (+   240us) service_pool.cc:224] Handling call
0709 13:19:55.420228 (+26933054us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4330,"reported_tablets":51}
I20260709 13:19:55.427318 16491 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 11 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:19:55.452466 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet fc0ed8d0a60640a0a5ce3a772c3eef5c (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:55.487275 16740 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:55.508610 21951 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:55.520668 17414 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:19:55.538988 21971 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:19:55.548727 22055 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:55.594964 22055 tablet_bootstrap.cc:492] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:19:55.615001 22055 ts_tablet_manager.cc:1403] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.399s	user 0.040s	sys 0.008s
I20260709 13:19:55.618207 21971 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 5 to 11, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:55.633865 16744 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 5 to 11, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:55.646654 22055 raft_consensus.cc:359] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:55.647701 22055 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:19:55.649720 17412 raft_consensus.cc:2955] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 5 to 11, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:55.673159 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ada6718f86ea49aa96bb0fdac1acca8e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:55.687251 22055 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:55.696040 16487 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 5 to 11, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:55.702005 22055 ts_tablet_manager.cc:1434] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.086s	user 0.007s	sys 0.000s
I20260709 13:19:55.704245 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-e0b4dc6fa23d44edb449d1be0842b184 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:19:55.704809 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-e0b4dc6fa23d44edb449d1be0842b184 on tablet e0b4dc6fa23d44edb449d1be0842b184 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:19:55.725267 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet ada6718f86ea49aa96bb0fdac1acca8e with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:55.742265 17412 raft_consensus.cc:1217] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Deduplicated request from leader. Original: 12.8->[12.9-12.9]   Dedup: 12.9->[]
I20260709 13:19:55.749472 22075 tablet_replica.cc:333] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:19:55.755522 22075 raft_consensus.cc:2243] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:55.756278 22075 raft_consensus.cc:2272] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:55.763912 16491 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 238) took 2390 ms. Trace:
I20260709 13:19:55.767371 16492 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.775444 16492 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:55.786520 16492 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 16 to 17, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.791049 22075 ts_tablet_manager.cc:1916] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:55.790979 16492 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 10 to 11, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 11 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:19:55.764410 16491 rpcz_store.cc:276] 0709 13:19:53.373264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:19:53.373456 (+   192us) service_pool.cc:224] Handling call
0709 13:19:55.763839 (+2390383us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:19:55.459377 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:19:55.459905 (+   528us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:19:55.459944 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:19:55.459961 (+    17us) tablet.cc:662] Decoding operations
0709 13:19:55.460933 (+   972us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:19:55.460987 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:19:55.461018 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:19:55.466536 (+  5518us) tablet.cc:701] Row locks acquired
0709 13:19:55.466553 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:19:55.466697 (+   144us) write_op.cc:270] Start()
0709 13:19:55.466820 (+   123us) write_op.cc:276] Timestamp: P: 1783603195466653 usec, L: 0
0709 13:19:55.466838 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:19:55.467576 (+   738us) log.cc:844] Serialized 3974 byte log entry
0709 13:19:55.493530 (+ 25954us) op_driver.cc:464] REPLICATION: finished
0709 13:19:55.495402 (+  1872us) write_op.cc:301] APPLY: starting
0709 13:19:55.495520 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:19:55.528027 (+ 32507us) tablet.cc:1370] finished BulkCheckPresence
0709 13:19:55.528169 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:19:55.610699 (+ 82530us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:19:55.672937 (+ 62238us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:19:55.675120 (+  2183us) write_op.cc:312] APPLY: finished
0709 13:19:55.675482 (+   362us) log.cc:844] Serialized 97 byte log entry
0709 13:19:55.677779 (+  2297us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:55.677935 (+   156us) write_op.cc:454] Released schema lock
0709 13:19:55.678003 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:19:55.678446 (+   443us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:19:55.678478 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39423,"reported_tablets":36,"child_traces":[["op",{"apply.queue_time_us":1565,"apply.run_cpu_time_us":16310,"apply.run_wall_time_us":183030,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41840,"num_ops":10,"prepare.queue_time_us":1608,"prepare.run_cpu_time_us":3834,"prepare.run_wall_time_us":8461,"replication_time_us":26569}]]}
I20260709 13:19:55.820145 22061 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=3 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:19:55.821347 22061 tablet_bootstrap.cc:492] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:19:55.822160 22061 ts_tablet_manager.cc:1403] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.534s	user 0.046s	sys 0.012s
I20260709 13:19:55.893785 16492 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet ada6718f86ea49aa96bb0fdac1acca8e with cas_config_opid_index 5: aborting the task: latest config opid_index 11; task opid_index 5
I20260709 13:19:55.866240 22061 raft_consensus.cc:359] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:55.896976 22061 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:19:55.897737 22061 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:55.900360 22061 ts_tablet_manager.cc:1434] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.000s
I20260709 13:19:55.909448 22075 ts_tablet_manager.cc:1929] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 13:19:55.909976 22075 log.cc:1199] T ada6718f86ea49aa96bb0fdac1acca8e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/ada6718f86ea49aa96bb0fdac1acca8e
I20260709 13:19:55.919809 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-f1a71b94ecfb4155a5cfe591077e9a1c received from {username='slave'} at 127.0.0.1:38524
I20260709 13:19:55.920362 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-f1a71b94ecfb4155a5cfe591077e9a1c on tablet f1a71b94ecfb4155a5cfe591077e9a1c with peer ba2df6a668144022a223e504ae571f21
I20260709 13:19:55.969774 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet ada6718f86ea49aa96bb0fdac1acca8e (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:56.036736 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:56.093459 17412 raft_consensus.cc:1217] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.10->[17.11-17.11]   Dedup: 17.11->[]
I20260709 13:19:56.354907 21971 raft_consensus.cc:1064] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:19:56.358301 21971 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:56.395262 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:56.395972 16665 raft_consensus.cc:606] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:56.403723 16737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:19:56.404395 16737 raft_consensus.cc:606] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:56.411653 22074 raft_consensus.cc:993] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:56.420078 16737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:19:56.420855 16737 raft_consensus.cc:606] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:56.426541 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:56.436229 22071 raft_consensus.cc:1081] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:56.469676 16666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36082
I20260709 13:19:56.470310 16666 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:56.470681 16666 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:56.484201 16666 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:56.502163 16666 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:56.506132 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:56.507153 17412 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:56.510983 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:56.511878 16744 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:56.512218 16744 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:56.512977 16744 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:56.514147 16744 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:56.526010 17412 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:56.527932 16665 raft_consensus.cc:606] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:56.528901 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:19:56.531402 21934 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:56.532042 21934 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:56.533064 21934 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:56.537986 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:19:56.541356 16737 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:56.546461 16595 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:56.552595 21951 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:56.556622 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:56.567965 21951 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:56.572369 17412 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:19:56.573992 16744 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
W20260709 13:19:56.619567 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 88496.1us
I20260709 13:19:56.684289 22083 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:56.688793 21951 raft_consensus.cc:1064] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:19:56.682220 16487 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:56.738271 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 54436.8us
W20260709 13:19:56.773320 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97920.9us
I20260709 13:19:56.766202 21951 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:56.864717 22071 raft_consensus.cc:993] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:56.865413 22071 raft_consensus.cc:1081] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
W20260709 13:19:56.868738 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 54494us
W20260709 13:19:56.914328 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 86221.7us
W20260709 13:19:56.917537 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 131309us
I20260709 13:19:56.928867 22093 raft_consensus.cc:993] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:56.932472 22080 raft_consensus.cc:1081] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:56.952770 21934 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Committing config change with OpId 12.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:56.969347 22091 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.146s	user 0.002s	sys 0.000s
W20260709 13:19:56.979019 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 62486.3us
W20260709 13:19:56.993610 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 81833.1us
I20260709 13:19:56.998656 16590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:56.999588 16590 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:57.000006 16590 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:19:57.011555 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 115172us
I20260709 13:19:57.022120 16744 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.027345 16739 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:57.034171 17414 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:57.042723 16590 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:57.044144 16595 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:19:57.050804 21934 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:57.056362 22102 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:57.057046 22083 raft_consensus.cc:993] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:57.057578 22102 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:57.057979 22102 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:57.058892 22102 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:57.060171 22102 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:57.056284 16595 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:19:57.063335 22083 raft_consensus.cc:1081] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:57.066830 16590 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:57.077142 22103 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:57.087168 16590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:57.088001 16590 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:57.088435 16590 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:57.105739 16590 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:57.111794 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:57.114318 16590 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:57.125399 16667 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:57.122392 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:57.134979 16665 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:19:57.135474 16665 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:57.136492 16665 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:57.139091 16665 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:19:57.141436 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 94549.9us
I20260709 13:19:57.143530 17412 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:57.145800 16744 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:57.150228 16492 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.161785 22102 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:57.171700 16665 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:57.173309 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:57.170610 22102 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:19:57.188634 22092 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:19:57.189452 22092 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:57.189957 22107 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:19:57.190721 22092 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:57.239118 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 325104us
I20260709 13:19:57.248443 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:57.249126 17410 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:57.255910 16595 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.261896 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:57.264487 16744 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:57.273746 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:57.274747 16739 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:19:57.250181 21951 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.412557 16739 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:19:57.415858 17410 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:57.439564 16744 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:19:57.431692 16492 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:57.453706 17410 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.512948 16493 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:57.522100 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 197714us
I20260709 13:19:57.527133 16595 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38582
I20260709 13:19:57.527674 16595 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:57.526705 16744 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Committing config change with OpId 17.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.527992 16595 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:57.540236 16595 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:57.542591 16594 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 9. Current term is 10. Ops: []
W20260709 13:19:57.551460 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 97540.3us
I20260709 13:19:57.562839 16595 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:57.563510 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:57.590533 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:57.592340 16738 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:19:57.592649 16738 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:57.593361 16738 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:57.594468 16738 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:57.594753 16561 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:57.595625 22078 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:57.596339 22078 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:57.615324 16738 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:57.597245 22078 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:57.642532 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60334.4us
I20260709 13:19:57.700892 16667 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:57.703380 21651 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:57.711328 16665 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:19:57.713893 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 107965us
I20260709 13:19:57.714936 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:19:57.743418 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 131949us
I20260709 13:19:57.732628 22118 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:19:57.755875 22118 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:57.756912 22118 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:57.781258 16595 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:57.787056 22082 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:57.802722 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:57.803349 17410 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:19:57.812656 17414 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:19:57.821677 22107 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.852591 17410 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:19:57.845541 16589 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.853638 22087 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:19:57.910524 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 179562us
I20260709 13:19:57.912806 16737 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:19:57.971637 17410 raft_consensus.cc:2955] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.892035 16493 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 10 to 9, term changed from 12 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from VOTER to NON_VOTER, now has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:57.986016 21651 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:19:58.000492 16492 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:58.013332 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f1a71b94ecfb4155a5cfe591077e9a1c with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:58.039851 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet f1a71b94ecfb4155a5cfe591077e9a1c with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:52804
I20260709 13:19:58.061625 22131 tablet_replica.cc:333] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:19:58.063400 22131 raft_consensus.cc:2243] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:19:58.064797 22131 raft_consensus.cc:2272] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:19:58.074654 22131 ts_tablet_manager.cc:1916] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:19:58.101943 16492 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 9 to 10, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 10 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:58.121439 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 129293us
I20260709 13:19:58.194473 22131 ts_tablet_manager.cc:1929] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.12
I20260709 13:19:58.195130 22131 log.cc:1199] T f1a71b94ecfb4155a5cfe591077e9a1c P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/f1a71b94ecfb4155a5cfe591077e9a1c
I20260709 13:19:58.195639 16590 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:58.196404 16590 raft_consensus.cc:606] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Received request to transfer leadership
I20260709 13:19:58.199687 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet f1a71b94ecfb4155a5cfe591077e9a1c (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:19:58.236726 16595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:58.237383 16595 raft_consensus.cc:606] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Received request to transfer leadership
W20260709 13:19:58.238651 16478 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for e0b4dc6fa23d44edb449d1be0842b184 already in progress [suppressed 1 similar messages]
I20260709 13:19:58.247572 16595 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:58.248256 16595 raft_consensus.cc:606] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Received request to transfer leadership
W20260709 13:19:58.253525 16542 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:19:58.253963 16542 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for e0b4dc6fa23d44edb449d1be0842b184 already in progress
W20260709 13:19:58.254249 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for e0b4dc6fa23d44edb449d1be0842b184 already in progress
I20260709 13:19:58.290318 16737 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:58.292904 22130 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:19:58.294137 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 69553us
I20260709 13:19:58.300020 17414 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:58.340277 17414 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:58.344755 22130 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:58.352164 22130 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:19:58.357739 16667 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:19:58.368278 22118 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:19:58.384954 16738 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:19:58.392151 16492 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1), VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2), VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3), VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:58.390461 22106 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:19:58.416054 22118 raft_consensus.cc:993] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:19:58.417443 22106 raft_consensus.cc:1081] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:19:58.421414 22125 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Committing config change with OpId 12.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:58.430127 16737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37636
I20260709 13:19:58.431201 16737 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:58.431717 16737 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:58.433924 17419 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Committing config change with OpId 12.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:58.496882 16737 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:58.543540 16742 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:19:58.555279 16737 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:19:58.557381 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:19:58.566812 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:58.568099 16665 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:58.573102 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:58.566286 22125 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:19:58.581642 16595 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:19:58.582000 16595 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:58.582753 16595 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:58.584081 16595 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:58.584789 16665 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:19:58.599706 16595 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
W20260709 13:19:58.600669 22118 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:19:58.625587 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:19:58.626617 22079 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:19:58.627225 22079 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:58.628206 22079 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:58.705585 16492 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 13 to 14, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:58.729902 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:58.935025 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3326ab1787714e708abfea6cf0509109"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:19:58.935662 16669 raft_consensus.cc:606] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:19:58.939275 16737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:19:58.939993 16737 raft_consensus.cc:606] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:19:58.944554 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:19:58.946913 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:19:59.104409 16665 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:59.117760 22119 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:59.186471 16591 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:59.188478 22119 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:19:59.213131 22071 raft_consensus.cc:993] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:59.213958 22071 raft_consensus.cc:1081] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:19:59.242316 16669 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36082
I20260709 13:19:59.242997 16669 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:59.243399 16669 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:59.248174 17410 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:19:59.256791 17410 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:19:59.315117 22079 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:19:59.325284 16669 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:59.328508 21934 raft_consensus.cc:993] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:59.329128 21934 raft_consensus.cc:1081] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:19:59.334602 16591 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3326ab1787714e708abfea6cf0509109"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:19:59.335263 16591 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:19:59.335652 16591 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:59.339130 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:19:59.339828 16595 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:59.354190 16595 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:59.355808 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:19:59.364141 21934 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:19:59.364776 21934 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:59.365695 21934 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:59.350407 16591 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:59.377923 16591 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:59.380162 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:19:59.380820 16665 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:19:59.388790 16665 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:19:59.389829 16665 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:59.380856 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:59.406672 16737 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:59.411777 16665 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:19:59.413971 16492 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:19:59.396656 16669 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:19:59.400626 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:19:59.419355 16739 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:19:59.419672 16739 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:19:59.420396 16739 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:19:59.421495 16739 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:19:59.418674 16737 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:59.440500 16739 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:19:59.440529 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:19:59.441854 22149 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:19:59.442417 22149 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:19:59.443394 22149 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:19:59.536489 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 82397us
I20260709 13:19:59.542436 16665 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 7.
I20260709 13:19:59.632225 16493 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 6 to 7, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 7 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:19:59.675035 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64448.9us
I20260709 13:19:59.740275 16737 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:19:59.754642 17414 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:59.757754 22119 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:19:59.766527 22079 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 10 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:59.775884 16590 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:19:59.777416 17414 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 10 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:59.785566 16742 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:19:59.787397 22107 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:59.793378 16669 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:19:59.802315 22071 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:19:59.808863 16669 raft_consensus.cc:2955] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 10 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:19:59.815573 16561 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d9aa91533ea94ae392057ef582e0c419. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:19:59.819362 16742 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:59.826051 22158 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:59.827399 16665 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:19:59.830369 22158 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:19:59.838995 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e0b4dc6fa23d44edb449d1be0842b184 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:19:59.898710 16738 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:19:59.903152 16493 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 10 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:19:59.908919 22149 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:19:59.948271 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:19:59.953127 22158 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Committing config change with OpId 9.7: config changed from index -1 to 7, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:19:59.963935 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet e0b4dc6fa23d44edb449d1be0842b184 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:42550
I20260709 13:19:59.965570 16739 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index -1 to 7, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:00.013933 16667 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:20:00.086817 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45689.3us
I20260709 13:20:00.096230 22176 tablet_replica.cc:333] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
W20260709 13:20:00.107465 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62792.1us
I20260709 13:20:00.109377 16590 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:20:00.119007 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 89337.9us
I20260709 13:20:00.109612 22176 raft_consensus.cc:2243] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:00.122674 22176 raft_consensus.cc:2272] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:00.134366 22166 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:00.135627 16667 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Committing config change with OpId 9.7: config changed from index -1 to 7, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:00.154989 16590 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:00.163215 22153 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:20:00.165824 16492 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 7, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:00.254065 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 106670us
I20260709 13:20:00.249004 22176 ts_tablet_manager.cc:1916] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:00.351888 16741 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:00.353219 16666 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:00.353612 22170 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:00.356006 22175 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:20:00.364408 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62412.7us
I20260709 13:20:00.377082 22176 ts_tablet_manager.cc:1929] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.12
I20260709 13:20:00.377713 22176 log.cc:1199] T e0b4dc6fa23d44edb449d1be0842b184 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/e0b4dc6fa23d44edb449d1be0842b184
I20260709 13:20:00.401309 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet e0b4dc6fa23d44edb449d1be0842b184 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
W20260709 13:20:00.425401 16561 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 51e96ec0851c4732b9a3e1272e9fd80f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.438809 22157 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Committing config change with OpId 14.12: config changed from index -1 to 12, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:00.475049 16666 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index -1 to 12, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:20:00.486552 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59090.1us
I20260709 13:20:00.487833 16737 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Committing config change with OpId 14.12: config changed from index -1 to 12, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:00.498222 16493 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 12, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:00.634965 16595 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:00.648437 22191 ts_tablet_manager.cc:933] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:20:00.669438 22191 tablet_copy_client.cc:323] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:20:00.671933 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet d9aa91533ea94ae392057ef582e0c419 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:20:00.672619 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet d9aa91533ea94ae392057ef582e0c419 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-d9aa91533ea94ae392057ef582e0c419
I20260709 13:20:00.676900 16739 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:00.677050 16665 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:20:00.678638 16561 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6b98cb4c3414b4e86be2fc2bcd9e2b5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:00.679226 22180 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:00.689913 22180 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:00.727718 16606 tablet_copy_source_session.cc:215] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:00.734957 22191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9aa91533ea94ae392057ef582e0c419. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:00.768827 22191 tablet_copy_client.cc:806] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:00.769647 22191 tablet_copy_client.cc:670] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:00.839718 22191 tablet_copy_client.cc:538] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:00.838239 22118 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Committing config change with OpId 20.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:00.954141 16669 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Committing config change with OpId 20.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:01.130102 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:01.171319 16737 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Committing config change with OpId 20.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:01.189041 22198 ts_tablet_manager.cc:933] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:20:01.270797 22198 tablet_copy_client.cc:323] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:20:01.301760 22191 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:01.318101 16492 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:20:01.325178 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59054.4us
I20260709 13:20:01.387205 16741 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:20:01.401837 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70419.7us
W20260709 13:20:01.465178 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63520.1us
I20260709 13:20:01.469493 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet 51e96ec0851c4732b9a3e1272e9fd80f from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:20:01.470351 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet 51e96ec0851c4732b9a3e1272e9fd80f from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-51e96ec0851c4732b9a3e1272e9fd80f
W20260709 13:20:01.470556 16710 consensus_peers.cc:597] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abd752e770ac43a799af500f16d446d5. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:01.471935 22209 ts_tablet_manager.cc:933] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:20:01.485729 16606 tablet_copy_source_session.cc:215] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:01.503499 16669 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:01.507618 22203 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:01.511613 16590 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:20:01.514480 22203 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:01.517560 22209 tablet_copy_client.cc:323] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:20:01.521752 22198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e96ec0851c4732b9a3e1272e9fd80f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:01.522159 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:20:01.525996 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-b6b98cb4c3414b4e86be2fc2bcd9e2b5
I20260709 13:20:01.575224 22199 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:01.579984 16666 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:01.606036 16606 tablet_copy_source_session.cc:215] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:01.629576 22209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:01.708065 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:01.709008 16664 raft_consensus.cc:606] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:01.713727 22198 tablet_copy_client.cc:806] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:01.714548 22198 tablet_copy_client.cc:670] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:01.727193 16493 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:01.731113 16591 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:01.765838 16587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:20:01.766494 16587 raft_consensus.cc:606] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:01.781200 22198 tablet_copy_client.cc:538] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:01.788221 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:01.790198 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:01.791486 22198 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:01.791987 22191 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:01.799816 22191 tablet_bootstrap.cc:492] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:01.800742 22191 ts_tablet_manager.cc:1403] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.499s	user 0.037s	sys 0.019s
I20260709 13:20:01.815948 22209 tablet_copy_client.cc:806] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:01.822687 22209 tablet_copy_client.cc:670] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:01.815007 22191 raft_consensus.cc:359] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:01.823822 22191 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:01.831437 22209 tablet_copy_client.cc:538] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:01.834085 22191 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:01.837270 22191 ts_tablet_manager.cc:1434] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:20:01.858803 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-d9aa91533ea94ae392057ef582e0c419 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:20:01.859457 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-d9aa91533ea94ae392057ef582e0c419 on tablet d9aa91533ea94ae392057ef582e0c419 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:01.893386 22209 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:02.022217 22198 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:02.023447 22198 tablet_bootstrap.cc:492] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:02.029098 22200 raft_consensus.cc:993] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:02.029994 21972 raft_consensus.cc:1081] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:02.038385 16741 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:02.039178 16741 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:02.039621 16741 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:02.062155 22191 ts_tablet_manager.cc:933] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:02.080947 17414 raft_consensus.cc:1217] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 LEARNER]: Deduplicated request from leader. Original: 9.6->[9.7-9.7]   Dedup: 9.7->[]
I20260709 13:20:02.097401 16741 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:02.110363 16741 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:02.083154 22198 ts_tablet_manager.cc:1403] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.292s	user 0.050s	sys 0.005s
I20260709 13:20:02.119627 22191 tablet_copy_client.cc:323] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:20:02.121836 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet abd752e770ac43a799af500f16d446d5 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:20:02.122251 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:02.122450 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet abd752e770ac43a799af500f16d446d5 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-abd752e770ac43a799af500f16d446d5
I20260709 13:20:02.122980 16666 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:02.123333 16666 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:02.124215 16666 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:02.125712 16666 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:02.130735 22198 raft_consensus.cc:359] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:02.131703 22198 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:02.132355 22198 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:02.142767 22209 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=3 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:02.143863 22209 tablet_bootstrap.cc:492] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:02.144661 22209 ts_tablet_manager.cc:1403] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.252s	user 0.042s	sys 0.015s
I20260709 13:20:02.146013 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:02.161427 16591 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:02.164983 17414 raft_consensus.cc:1217] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.11->[14.12-14.12]   Dedup: 14.12->[]
I20260709 13:20:02.185216 22198 ts_tablet_manager.cc:1434] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 13:20:02.191105 16754 tablet_copy_source_session.cc:215] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:02.195652 16666 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:02.218060 16591 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:02.219673 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:02.220697 22199 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:02.221201 22191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd752e770ac43a799af500f16d446d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:02.221287 22199 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:02.205621 22209 raft_consensus.cc:359] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:02.222484 22199 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:02.223104 22209 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:02.224102 22209 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:02.227553 22209 ts_tablet_manager.cc:1434] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.002s
W20260709 13:20:02.244791 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75958.5us
I20260709 13:20:02.272791 16493 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 11 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:02.293635 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-51e96ec0851c4732b9a3e1272e9fd80f received from {username='slave'} at 127.0.0.1:47022
I20260709 13:20:02.294204 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-51e96ec0851c4732b9a3e1272e9fd80f on tablet 51e96ec0851c4732b9a3e1272e9fd80f with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:02.313759 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-b6b98cb4c3414b4e86be2fc2bcd9e2b5 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:20:02.314263 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-b6b98cb4c3414b4e86be2fc2bcd9e2b5 on tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:02.317209 22191 tablet_copy_client.cc:806] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:02.321920 22191 tablet_copy_client.cc:670] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:02.343770 22220 raft_consensus.cc:993] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:02.344681 22191 tablet_copy_client.cc:538] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:02.352063 22220 raft_consensus.cc:1081] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:02.367340 16737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37636
I20260709 13:20:02.367980 16737 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:02.368311 16737 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:02.380511 16737 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:02.391911 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:02.392535 17414 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:02.393352 22191 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:02.399026 16737 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:02.399506 17414 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:02.400206 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:02.400837 16587 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:02.401140 16587 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:02.402338 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:02.403333 22208 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:02.402100 16587 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:02.403843 22208 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:02.404484 16587 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:02.405084 22208 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:02.421221 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:02.437841 16493 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 10 to 11, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 11 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:02.565661 17419 raft_consensus.cc:1217] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.10->[20.11-20.11]   Dedup: 20.11->[]
I20260709 13:20:02.592486 22191 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=2 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:02.593631 22191 tablet_bootstrap.cc:492] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:02.594368 22191 ts_tablet_manager.cc:1403] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.201s	user 0.045s	sys 0.003s
I20260709 13:20:02.597748 22191 raft_consensus.cc:359] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:02.598547 22191 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:02.603446 22191 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:02.634692 22118 raft_consensus.cc:1064] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:02.640055 16595 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:02.647459 22191 ts_tablet_manager.cc:1434] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:20:02.652339 22203 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:02.637490 22118 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 10, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:02.659116 22205 raft_consensus.cc:1064] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:02.662467 16754 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-abd752e770ac43a799af500f16d446d5 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:20:02.663035 16754 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-abd752e770ac43a799af500f16d446d5 on tablet abd752e770ac43a799af500f16d446d5 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:02.676788 16666 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:02.680409 16741 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:02.682772 17419 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:02.685735 22220 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:02.691555 22220 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:02.670092 22205 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:02.731041 22118 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:02.772814 17414 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:02.777159 22170 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:02.784788 16741 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:02.791625 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:02.803354 22118 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:20:02.807648 22231 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:20:02.812235 16666 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:20:02.815623 22205 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:02.826838 17414 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:02.832041 22199 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:02.853706 16666 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:20:02.858948 22208 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:02.867141 22205 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Committing config change with OpId 14.13: config changed from index 12 to 13, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:02.875711 16590 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:20:02.889796 17419 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:02.897395 16741 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:03.147192 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 154043us
I20260709 13:20:03.157907 22248 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:03.162042 16493 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 12 to 13, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:03.179683 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 91486.5us
I20260709 13:20:03.167104 16663 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Committing config change with OpId 14.13: config changed from index 12 to 13, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:03.245405 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 73517.5us
I20260709 13:20:03.259686 17419 raft_consensus.cc:1217] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.10->[15.11-15.11]   Dedup: 15.11->[]
I20260709 13:20:03.260435 22205 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Committing config change with OpId 9.8: config changed from index 7 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:03.293309 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 217408us
W20260709 13:20:03.308180 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 72219.1us
I20260709 13:20:03.333464 16668 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:03.378731 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 206666us
I20260709 13:20:03.383229 17414 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.394745 16737 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Committing config change with OpId 9.8: config changed from index 7 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.460376 22261 raft_consensus.cc:1064] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:03.465528 22261 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 9, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:03.551182 16489 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 7 to 8, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.578751 16587 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:03.589814 16663 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:20:03.611523 22251 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:03.622150 17419 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:20:03.622778 16744 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:03.624954 17419 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:03.626222 16668 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:03.635895 22229 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:03.637249 22251 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:03.638798 22220 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Committing config change with OpId 14.14: config changed from index 13 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.646061 16663 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Committing config change with OpId 14.14: config changed from index 13 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.653159 22241 raft_consensus.cc:1064] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:03.657384 22241 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:03.667672 22229 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:03.671209 22251 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:03.673004 17419 raft_consensus.cc:2955] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.14: config changed from index 13 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.699780 16487 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 13 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.753924 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet 51e96ec0851c4732b9a3e1272e9fd80f with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:52804
I20260709 13:20:03.753159 22251 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Committing config change with OpId 20.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.757879 16668 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.776773 17414 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:03.778599 22208 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:03.782826 16737 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.794999 22271 tablet_replica.cc:333] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:20:03.795712 16589 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:03.800156 22271 raft_consensus.cc:2243] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:03.798623 16668 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:03.802186 22208 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:03.820813 22208 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:03.805140 16492 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.828037 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 51e96ec0851c4732b9a3e1272e9fd80f with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:03.879201 22271 raft_consensus.cc:2272] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:03.903038 16589 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:03.924429 22271 ts_tablet_manager.cc:1916] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:03.930335 17414 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:03.936995 22241 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:03.941589 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:03.956525 22229 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:03.961512 16737 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:20:03.971012 22271 ts_tablet_manager.cc:1929] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.13
I20260709 13:20:04.005347 22271 log.cc:1199] T 51e96ec0851c4732b9a3e1272e9fd80f P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/51e96ec0851c4732b9a3e1272e9fd80f
I20260709 13:20:04.007679 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet 51e96ec0851c4732b9a3e1272e9fd80f (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:04.026648 22258 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.044312 17414 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Committing config change with OpId 20.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.044504 16589 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.050091 22220 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:04.060464 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d9aa91533ea94ae392057ef582e0c419 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:04.062808 16739 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.063833 16591 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.068984 16487 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 8 to 9, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.064510 17419 raft_consensus.cc:2955] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Committing config change with OpId 9.9: config changed from index 8 to 9, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.079244 16666 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:04.081064 22220 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:04.087651 17414 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:20:04.088730 22258 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.094009 16663 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.111464 16668 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.119067 22251 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:04.127197 17414 raft_consensus.cc:2955] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.165566 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:04.167533 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet d9aa91533ea94ae392057ef582e0c419 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 9) from {username='slave'} at 127.0.0.1:59566
I20260709 13:20:04.177826 16492 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 12 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.235378 17414 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.247048 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:52804
I20260709 13:20:04.248099 22271 tablet_replica.cc:333] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:20:04.249167 22271 raft_consensus.cc:2243] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.249963 22271 raft_consensus.cc:2272] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.254739 16739 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:20:04.255295 22271 ts_tablet_manager.cc:1916] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:04.255486 16739 raft_consensus.cc:606] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:04.259795 16739 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:20:04.260514 16739 raft_consensus.cc:606] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:04.260772 22277 tablet_replica.cc:333] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:20:04.262233 22277 raft_consensus.cc:2243] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.263293 22277 raft_consensus.cc:2272] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.229328 16487 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:04.265228 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:04.310950 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:04.306318 16739 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.275736 22277 ts_tablet_manager.cc:1916] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:04.324589 17414 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:04.324751 16668 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:04.327370 22208 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:04.336303 22199 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:04.346267 22246 raft_consensus.cc:993] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:20:04.346910 22248 raft_consensus.cc:1081] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:20:04.350927 22203 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.352524 17419 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.355937 16666 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36082
I20260709 13:20:04.356535 16666 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:04.356902 16666 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.364059 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet abd752e770ac43a799af500f16d446d5 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:04.364398 16668 raft_consensus.cc:2955] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Committing config change with OpId 15.13: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.377467 16666 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.383206 16666 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:04.387348 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:04.412840 17410 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.420672 22208 raft_consensus.cc:993] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:04.421216 22208 raft_consensus.cc:1081] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:04.391304 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:04.423115 16591 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38582
I20260709 13:20:04.423213 16739 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:04.423667 16591 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:04.424044 16591 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:04.423686 16739 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:04.430660 17410 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:04.433871 16492 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:04.435992 16739 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.435473 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:04.440723 21972 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:04.441238 21972 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:04.441318 16739 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:04.442057 21972 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.448899 16739 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:04.470270 22271 ts_tablet_manager.cc:1929] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.12
I20260709 13:20:04.470851 22271 log.cc:1199] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/b6b98cb4c3414b4e86be2fc2bcd9e2b5
I20260709 13:20:04.472323 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet b6b98cb4c3414b4e86be2fc2bcd9e2b5 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:04.462092 16591 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.476126 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet abd752e770ac43a799af500f16d446d5 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:42550
I20260709 13:20:04.487742 16591 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:04.488720 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:04.490813 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:04.496207 17410 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:04.498831 16739 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:04.502918 17410 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:04.499176 16739 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:04.504495 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:04.520649 16739 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.521862 16739 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:04.518958 22258 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:04.527370 22258 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:04.527868 16739 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:04.528316 22258 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:04.515738 16489 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 11 to 12, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:04.531381 22277 ts_tablet_manager.cc:1929] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 9.8
I20260709 13:20:04.531795 22277 log.cc:1199] T d9aa91533ea94ae392057ef582e0c419 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/d9aa91533ea94ae392057ef582e0c419
I20260709 13:20:04.537312 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet d9aa91533ea94ae392057ef582e0c419 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:04.546756 22279 tablet_replica.cc:333] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:20:04.547989 22279 raft_consensus.cc:2243] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:04.547838 16492 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:04.567293 22279 raft_consensus.cc:2272] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:04.584597 22279 ts_tablet_manager.cc:1916] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:04.638638 22279 ts_tablet_manager.cc:1929] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.12
I20260709 13:20:04.660602 22279 log.cc:1199] T abd752e770ac43a799af500f16d446d5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/abd752e770ac43a799af500f16d446d5
I20260709 13:20:04.662590 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet abd752e770ac43a799af500f16d446d5 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:04.829627 16737 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:04.832826 22200 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:04.962723 17419 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:04.964299 22200 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:05.019524 17410 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:05.024794 22220 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:20:05.035121 16737 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:05.041237 22220 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:05.339953 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:05.653874 16587 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:20:05.698977 16561 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d104b6d16beb411c9099f23901cebbb2. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:05.709664 16737 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:05.712641 22220 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:05.722522 16668 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:05.724287 22220 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:05.777225 22251 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:05.793676 16737 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:05.799234 16668 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Committing config change with OpId 14.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:05.817989 16492 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:05.847477 16737 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:20:05.899336 16710 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ecf0f1df23742a2a06e910e6b7350a0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:05.903108 16668 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:05.905795 22199 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:05.958261 16587 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:05.965309 22208 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:06.097297 22199 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:06.474503 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:07.091130 22306 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.092033 22306 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.106278 22220 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.111930 22220 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.097965 22306 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.111209 22229 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.114017 22220 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.113919 22229 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.116169 22229 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.122321 22229 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:07.123101 22229 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.127643 22307 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.128557 22229 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.128506 22307 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.130839 22307 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.140058 22307 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.150995 21972 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.156459 21972 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.156275 22307 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.158914 21972 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.160353 22229 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.160956 22229 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.163369 22229 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.159889 22307 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.166152 22310 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.167227 22310 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.169785 22310 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.174958 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.175750 16668 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:07.176432 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2211) took 1011 ms. Trace:
I20260709 13:20:07.176733 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.176880 16737 rpcz_store.cc:276] 0709 13:20:06.164772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:07.175694 (+1010922us) service_pool.cc:224] Handling call
0709 13:20:07.175972 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:07.176128 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:07.176151 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:07.176187 (+    36us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:07.176226 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.176275 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:07.176363 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:07.177822 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.178908 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.179965 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.182974 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:07.184212 22307 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:07.185312 22307 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.186606 22307 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.200623 22307 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.203544 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:07.204129 16663 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.206344 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.212061 16663 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:07.216940 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:07.217252 22307 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.217898 22306 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:07.218586 22306 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:07.219588 22306 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.234072 22309 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.234716 22309 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.236824 22309 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.246752 22307 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.247403 22307 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:07.248090 16710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.235s	user 0.000s	sys 0.029s
W20260709 13:20:07.248518 16710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.000s	sys 0.029s
W20260709 13:20:07.248813 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 1.23599e+06us
I20260709 13:20:07.249749 22307 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.261945 16489 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:07.275357 22306 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.276064 22306 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.280122 22306 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.282876 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.280318 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 2953) took 1311 ms. Trace:
I20260709 13:20:07.282214 22309 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.282533 22313 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.283978 22313 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.283648 16587 rpcz_store.cc:276] 0709 13:20:05.969089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:05.969210 (+   121us) service_pool.cc:224] Handling call
0709 13:20:05.969661 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:05.969992 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:05.970018 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:05.970851 (+   833us) log.cc:844] Serialized 492 byte log entry
0709 13:20:07.279814 (+1308963us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54002ad8a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fb5b6b8 00007f327fbe492b 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c
0709 13:20:07.279873 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:07.279917 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:07.279983 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:07.280024 (+    41us) raft_consensus.cc:1685] finished
0709 13:20:07.280034 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:07.280231 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292734208,"thread_start_us":132032,"threads_started":1,"wal-append.queue_time_us":129808}
I20260709 13:20:07.287098 22309 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.289372 22309 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.289832 22313 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.292250 21972 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.292639 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.292821 21972 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.293342 16589 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:07.294761 21972 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.297096 22306 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.297770 22306 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.298141 16662 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:07.300566 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:07.303746 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.305063 22200 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:07.305442 22200 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.305768 22200 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:07.307940 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.299s	user 0.022s	sys 0.054s
W20260709 13:20:07.308413 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.022s	sys 0.054s
I20260709 13:20:07.301323 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:07.308714 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 1.29984e+06us
I20260709 13:20:07.313169 16670 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:07.321130 22258 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.321560 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:07.322175 16667 raft_consensus.cc:606] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.321756 22258 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.322362 22307 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.323582 22208 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:07.323652 22307 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.324967 22258 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.326715 22307 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.330206 22200 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.325387 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.324324 22208 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.344110 22301 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:07.355917 22314 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.356173 22301 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.339996 22319 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.357318 22319 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.369063 22319 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.377589 22319 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.354966 22315 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.355082 22311 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.358474 22301 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.379532 16670 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:07.380061 22331 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.380682 22331 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.381703 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.379249 22311 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.384370 22311 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.359609 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:07.360319 22309 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.344848 22306 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.385290 22309 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.387195 22331 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.387650 22309 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.365943 22323 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:07.366647 16589 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:07.366881 22200 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.348730 22317 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.392342 22317 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.392737 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.393373 22322 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:07.394085 22320 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.374810 22307 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.375459 16591 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:20:07.395004 22317 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.395141 16591 raft_consensus.cc:606] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.394753 22307 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.390095 22315 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.377635 22316 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.356498 22314 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.363010 22208 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.391273 22309 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:07.393894 22322 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.403224 22318 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.403565 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.404254 16591 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:07.404974 22322 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.405545 22322 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.406327 22306 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.406051 22322 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.398320 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.352746 22326 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.399257 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:07.407567 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.397442 22307 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.398028 22316 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.399832 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.417894 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.399745 22320 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.419140 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.420341 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.420631 22320 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.421159 17419 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:07.421314 22208 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.400774 22311 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.401402 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.403878 22318 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.426051 22311 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.426685 22318 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.409330 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:07.395078 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.406848 22306 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.428364 22311 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.428463 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.429162 17412 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:07.402366 22315 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.429839 22306 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.430109 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.430771 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.431396 17416 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:07.416150 22332 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.416757 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.432632 17410 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:07.389909 22323 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.432157 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.416685 22326 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.427783 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.429050 22314 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.441322 22327 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:07.434233 17419 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:20:07.432135 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.422281 22208 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.447201 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.473222 17419 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:07.469537 16492 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:07.448228 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.448318 22332 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.449291 22330 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:07.450031 22324 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.454097 16592 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:07.455201 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.455896 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.446633 22301 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.462844 22323 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.485098 22324 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.463878 22322 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.432215 22315 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.470283 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.470594 16595 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:07.474203 16737 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:07.475911 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.477283 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.487277 22324 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.482247 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.487588 22315 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.482493 22336 ts_tablet_manager.cc:933] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:20:07.482525 22301 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.494228 22208 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.475509 22330 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:07.495949 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.496906 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.497196 22330 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.499183 22332 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.501091 22301 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.498694 22330 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:07.407229 22319 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:07.503335 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.504904 22316 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.504993 22326 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.513729 16591 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:07.515797 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.517409 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.518179 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.519858 22319 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.520903 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.506078 22309 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:07.534224 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:07.516666 22326 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.535358 22326 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.535677 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.536898 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.537431 22326 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.510241 22301 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.529656 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:07.538228 22301 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.539821 22307 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:07.540475 22307 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.541075 22307 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:07.551887 22315 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.552752 22313 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:07.553298 16636 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8b0d179ad714b59b34ab1dc57a57c02. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:07.553941 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:07.554693 22200 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:07.555174 22200 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.555542 22200 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.556737 16737 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:07.557710 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.540346 22301 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.562942 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.563617 22208 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:07.564502 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.565730 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:07.566519 22208 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:07.566848 22208 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.567171 22208 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:07.567730 22307 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.570997 22301 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.571556 22301 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.573438 22301 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.577113 22208 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.578310 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:07.578514 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.578976 22208 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.579099 22306 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:07.579463 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:07.580238 22306 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:07.580365 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:07.580636 22306 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.581033 22306 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.581430 22307 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.581521 22200 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.583756 16737 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:07.583978 22229 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:07.584481 22200 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.585083 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:07.591722 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.592341 16741 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:20:07.594710 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.595548 22315 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.597951 16737 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:07.598429 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:07.599074 17409 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:07.600966 22306 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.606827 17409 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:07.607975 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:07.609656 22307 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:07.609755 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:07.610481 22307 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:07.610514 17409 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.611573 22307 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.608453 22306 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.617650 17409 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:07.625808 22336 tablet_copy_client.cc:323] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:20:07.627655 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:07.623332 16489 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:07.554780 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:20:07.631733 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66074.9us
W20260709 13:20:07.633793 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 79073.4us
I20260709 13:20:07.633886 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.634392 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:07.634562 22208 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.635329 22208 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.637318 16741 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:07.639626 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.640044 16670 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:07.640008 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.641214 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.642371 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.642506 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.643203 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:07.643360 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:07.644275 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.644780 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.644785 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.645799 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:07.646363 17409 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.646711 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.648084 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.648241 22330 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:07.648644 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.644119 22310 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:07.649152 22310 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.649462 22308 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:07.650043 22308 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.650382 22308 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:07.650926 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.646744 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.654212 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.649519 22310 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.663556 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.663781 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:07.664351 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.665123 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.644439 22306 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:07.665853 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.666354 22337 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.666702 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.667060 22330 consensus_queue.cc:1243] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:07.681015 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.680465 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.679409 17409 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:07.678445 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.638448 22301 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:07.681053 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.685763 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.686709 22329 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:07.687036 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.687351 22329 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.688227 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.689379 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.686573 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet d104b6d16beb411c9099f23901cebbb2 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:20:07.689604 22329 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:07.689980 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet d104b6d16beb411c9099f23901cebbb2 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-d104b6d16beb411c9099f23901cebbb2
I20260709 13:20:07.690430 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.695472 22315 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:07.697839 16606 tablet_copy_source_session.cc:215] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:07.698951 22337 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.699929 22330 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:07.699221 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.701126 22315 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.701263 22337 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.703329 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.704427 22301 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:07.683885 22306 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:07.708046 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.707937 22306 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.708845 22330 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.710284 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.711066 22330 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:07.712479 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.713219 22330 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.699116 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.716148 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.716441 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.716715 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.717234 16741 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:07.717935 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.718012 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:07.718766 16745 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:07.719249 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:07.719533 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.720227 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.720361 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:07.720571 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.721307 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.721457 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:07.733322 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.734201 22208 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:07.734239 17420 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:07.783753 22315 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:07.784379 22315 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.788986 22336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d104b6d16beb411c9099f23901cebbb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:07.794611 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.796224 17420 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:07.830685 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.832825 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:07.835285 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.837589 17409 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
W20260709 13:20:07.839535 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45024us
I20260709 13:20:07.841540 22208 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:07.857966 16744 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:07.858587 16744 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:07.859184 16662 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:07.860811 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.832422 22315 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:07.863930 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:07.843413 22322 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:07.864389 21972 raft_consensus.cc:993] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
W20260709 13:20:07.870716 22334 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:20:07.882012 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.884572 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.884626 21972 raft_consensus.cc:1081] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:07.870734 17376 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.886957 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:07.890538 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46038us
I20260709 13:20:07.891971 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.892061 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.830423 22308 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.896314 17376 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.923729 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:07.894452 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:07.924907 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.925170 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.921288 16668 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:07.897486 22320 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.880903 16744 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:07.923061 22307 raft_consensus.cc:993] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:07.923090 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.896314 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:07.916831 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.931656 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:07.929975 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50164.8us
I20260709 13:20:07.866005 22318 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:07.930104 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:07.933737 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:07.940519 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 80241.7us
I20260709 13:20:07.942940 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:07.940143 22307 raft_consensus.cc:1081] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
W20260709 13:20:07.945041 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48802.9us
I20260709 13:20:07.930729 22332 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.941773 16744 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:07.945978 21972 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:07.951308 21972 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.951715 21972 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:07.948589 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.949054 22200 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:07.949285 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:07.953033 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.935240 22331 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.955188 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:07.955543 22200 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:07.955955 22200 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:07.956295 22200 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:07.941344 22306 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:07.947207 22326 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:07.950222 22322 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:07.984062 22323 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:07.987726 22318 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:07.987207 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43929.3us
I20260709 13:20:07.958211 22315 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:08.000552 22200 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:07.940719 22329 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:20:08.005649 22200 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:08.009677 22346 ts_tablet_manager.cc:933] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:08.019217 22310 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.019155 22347 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:20:08.021869 16671 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:08.023846 22310 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:08.024478 22329 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:20:08.059038 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 109884us
I20260709 13:20:08.059957 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:08.061692 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 164878us
W20260709 13:20:08.062813 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 168164us
I20260709 13:20:07.996955 22301 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:20:08.007292 22324 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:20:08.079586 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.061257 22329 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:08.080695 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.083925 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.090063 22354 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:08.090705 22354 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:08.091071 22330 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:08.091619 22354 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:08.094929 22308 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:08.096487 22307 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:08.102348 22337 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:08.102880 22337 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:08.103570 22320 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:08.104005 22320 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:08.103829 22337 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.104905 22320 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.106096 22320 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:08.092191 22330 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:08.107156 22330 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:08.108161 22330 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.108487 22258 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:08.109493 22330 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:08.110404 16670 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:08.111057 22333 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:08.111471 22333 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:08.111413 21972 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.112506 22333 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.115047 22333 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:08.122061 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:08.137351 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56749.5us
I20260709 13:20:08.137356 16744 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:08.134799 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.142326 21972 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:08.144939 22344 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:08.145704 22344 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:08.146847 22344 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.139968 22258 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:08.158955 22319 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:20:08.160023 22329 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.166333 22329 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:08.173101 16740 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:08.173902 16740 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:08.174326 16740 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:08.178177 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:08.179848 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:08.181512 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:08.187422 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:08.195760 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:08.195832 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:20:08.196748 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 240352us
I20260709 13:20:08.193956 16489 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 11 to 12, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 12 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.198060 17376 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.200196 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:08.201056 16489 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.204316 17376 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.165436 22346 tablet_copy_client.cc:323] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
W20260709 13:20:08.191597 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002efcc0 after lost signal to thread 15479
W20260709 13:20:08.208901 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 254520us
W20260709 13:20:08.209339 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002eac80 after lost signal to thread 16528
W20260709 13:20:08.209200 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 74452.2us
I20260709 13:20:08.210403 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:08.210563 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002df480 after lost signal to thread 16620
W20260709 13:20:08.211845 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08002dec20 after lost signal to thread 16695
W20260709 13:20:08.214393 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08003c5aa0 after lost signal to thread 16769
W20260709 13:20:08.216060 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08003098e0 after lost signal to thread 17446
I20260709 13:20:08.212162 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:08.218528 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.217489 22220 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:08.208189 22311 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:08.202888 22318 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:08.239171 22319 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:08.216760 22316 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:20:08.280741 22258 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:08.239878 16710 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:20:08.291786 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52032.7us
I20260709 13:20:08.292840 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.235618 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:08.295490 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60003.4us
I20260709 13:20:08.296576 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.235251 22337 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:08.240280 16739 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:08.244689 22220 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:08.306177 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.302851 16710 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.308239 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.276989 22336 tablet_copy_client.cc:806] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:08.311522 16710 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.309206 16740 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:08.317741 16738 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:08.320488 16664 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:08.322971 16710 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.303970 22326 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:08.304139 22324 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:08.310627 22319 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:08.326494 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 122902us
I20260709 13:20:08.327199 16740 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:08.311024 22229 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:08.310822 22347 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:08.329962 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:08.324142 22336 tablet_copy_client.cc:670] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:08.333267 16664 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:08.333616 16664 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:08.324018 22316 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:08.335891 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.338678 16664 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:08.348188 22350 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:08.348415 22358 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:08.352056 22313 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:08.337581 22330 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:08.362926 16664 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:08.366153 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.366524 16737 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:08.395671 16742 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:08.395725 22208 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:08.396397 22208 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:08.397199 22208 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.397840 22366 ts_tablet_manager.cc:933] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:08.398281 22208 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:08.405625 22220 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:08.428743 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:08.437268 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 114445us
W20260709 13:20:08.444785 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 98119us
I20260709 13:20:08.445137 22366 tablet_copy_client.cc:323] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
W20260709 13:20:08.447216 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 137053us
I20260709 13:20:08.413239 16664 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:08.449399 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:08.467834 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.471927 16743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37636
I20260709 13:20:08.473093 16743 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:08.473625 16743 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:08.474357 22369 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:08.472116 22337 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:08.479353 22326 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:08.480239 22326 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:20:08.491530 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.491425 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 0ecf0f1df23742a2a06e910e6b7350a0 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:20:08.492194 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 0ecf0f1df23742a2a06e910e6b7350a0 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-0ecf0f1df23742a2a06e910e6b7350a0
I20260709 13:20:08.494146 16745 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:08.502178 17409 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:20:08.507956 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 172113us
I20260709 13:20:08.509796 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:08.512454 22310 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:08.517251 16742 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:08.525478 16754 tablet_copy_source_session.cc:215] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:08.527305 16737 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:08.554693 16743 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.557188 16740 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:08.559075 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.559855 21972 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:08.560231 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:08.560322 21972 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:08.570096 16743 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:08.570467 21972 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.576071 16664 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:08.481462 22326 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:08.567430 22354 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:08.571586 22323 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:20:08.584172 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet e8b0d179ad714b59b34ab1dc57a57c02 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
W20260709 13:20:08.581408 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:08.586234 22346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecf0f1df23742a2a06e910e6b7350a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:08.593453 22369 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:08.595023 22354 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:08.596055 22354 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.592841 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet e8b0d179ad714b59b34ab1dc57a57c02 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-e8b0d179ad714b59b34ab1dc57a57c02
I20260709 13:20:08.605265 16595 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:08.613125 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:08.613276 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:08.613870 16594 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:08.613950 17415 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:08.613276 16743 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:08.616083 22320 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:08.616492 16591 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:08.617208 22320 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:08.617616 22320 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:08.618533 22320 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.620168 22320 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:08.631672 17415 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:20:08.632433 16594 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:08.632874 16710 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:08.636337 17415 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:08.639695 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:08.640331 16594 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:08.640705 16594 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:08.641731 16594 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.647743 16594 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:08.655946 22362 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:08.659390 22329 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:08.660382 16591 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:08.662812 22356 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:08.671767 22346 tablet_copy_client.cc:806] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:08.671901 22356 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:08.672457 22346 tablet_copy_client.cc:670] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:08.672958 22356 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.684531 16670 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:08.685447 22200 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:08.674603 22356 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:08.696211 17419 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:08.698475 16493 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 14, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:08.703166 16736 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:20:08.705488 16710 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b778f817264440987b35333e6540c72. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:08.701042 22346 tablet_copy_client.cc:538] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:08.684953 22329 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:08.714074 16594 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:20:08.741236 22348 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:08.742545 22329 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:08.743511 22348 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:08.759145 16594 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:08.765802 22310 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:08.766431 16681 tablet_copy_source_session.cc:215] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:08.751292 22314 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:20:08.756007 22313 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:20:08.780628 22366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8b0d179ad714b59b34ab1dc57a57c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:08.803498 22336 tablet_copy_client.cc:538] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:08.877875 16587 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:08.904527 22346 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:08.955804 16489 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:08.973910 22334 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:20:08.944866 22352 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:20:09.047613 22381 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:20:09.050784 22336 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:09.082268 22375 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:09.104507 22358 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:09.127220 16664 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:09.132066 16670 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:09.133770 22326 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:09.138345 16740 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:09.145447 22355 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:20:09.185765 17409 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:09.188500 17415 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:09.196816 16590 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Committing config change with OpId 16.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:09.208101 17409 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:09.214578 21972 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:20:09.227689 16489 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 14 to 15, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:09.235812 16489 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:09.255218 17415 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:20:09.250946 22394 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:09.264532 22400 ts_tablet_manager.cc:933] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:09.280603 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84456.8us
I20260709 13:20:09.292968 17415 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:20:09.340818 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51210.9us
I20260709 13:20:09.258822 22355 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:20:09.351251 16737 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:09.382236 16595 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:09.336889 22329 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:20:09.398535 16596 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:09.378698 22309 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:20:09.426541 22375 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:09.427155 22366 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48618 (request call id 1009) took 1251 ms. Trace:
W20260709 13:20:09.443657 22405 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:20:09.411654 22400 tablet_copy_client.cc:323] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:20:09.446033 22375 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:20:09.427608 22366 rpcz_store.cc:276] 0709 13:20:08.175957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:08.177749 (+  1792us) service_pool.cc:224] Handling call
0709 13:20:08.398207 (+220458us) ts_tablet_manager.cc:934] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
0709 13:20:08.778774 (+380567us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:09.082615 (+303841us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:09.082734 (+   119us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:09.426419 (+343685us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:09.426718 (+   299us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:09.427012 (+   294us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:09.427077 (+    65us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":591,"tablet-copy.queue_time_us":159595,"thread_start_us":175189,"threads_started":2}
I20260709 13:20:09.450109 22366 tablet_copy_client.cc:806] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:09.450780 22366 tablet_copy_client.cc:670] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
W20260709 13:20:09.478652 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 133413us
W20260709 13:20:09.489044 22409 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.084s	user 0.000s	sys 0.003s
I20260709 13:20:09.559293 16754 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 8b778f817264440987b35333e6540c72 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:20:09.559929 16754 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 8b778f817264440987b35333e6540c72 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-8b778f817264440987b35333e6540c72
I20260709 13:20:09.645664 16754 tablet_copy_source_session.cc:215] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:09.656248 22366 tablet_copy_client.cc:538] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:09.666442 22400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b778f817264440987b35333e6540c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:09.674021 16493 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 17 to 18, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:09.675726 16493 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 9 to 10, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 10 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:09.729497 22366 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:09.757251 16492 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:09.816874 22346 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:09.818032 22346 tablet_bootstrap.cc:492] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:09.832484 22346 ts_tablet_manager.cc:1403] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.943s	user 0.040s	sys 0.026s
I20260709 13:20:09.839684 22346 raft_consensus.cc:359] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:09.841702 22346 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:09.842520 22346 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:09.852962 22346 ts_tablet_manager.cc:1434] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:20:09.858724 22400 tablet_copy_client.cc:806] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:09.859419 22400 tablet_copy_client.cc:670] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:09.861507 16754 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-0ecf0f1df23742a2a06e910e6b7350a0 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:20:09.862020 16754 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-0ecf0f1df23742a2a06e910e6b7350a0 on tablet 0ecf0f1df23742a2a06e910e6b7350a0 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:09.866568 22400 tablet_copy_client.cc:538] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:09.877785 17420 raft_consensus.cc:1217] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Deduplicated request from leader. Original: 12.10->[12.11-12.11]   Dedup: 12.11->[]
I20260709 13:20:09.896860 22311 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:09.907200 22311 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:09.909545 22311 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:09.909967 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:09.910653 16590 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:09.910737 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:09.911438 16736 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:09.911973 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:09.912793 22311 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:09.913722 22311 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:09.914527 22311 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:09.917268 22400 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:09.954459 22311 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:09.964802 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:09.965411 16590 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:09.967182 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:09.967752 16736 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:09.975778 22336 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:09.989995 22336 tablet_bootstrap.cc:492] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:09.992323 22337 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:09.996069 22337 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:09.998812 22337 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.001266 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:10.002923 17420 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:10.004221 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.004957 22337 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:10.005645 22337 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:10.006110 22337 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:10.006477 16736 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:10.000509 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:10.008154 16671 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:10.008116 17376 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.009145 22397 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:10.009672 22397 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:10.010509 22397 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.006922 16590 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:10.015652 22336 ts_tablet_manager.cc:1403] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.965s	user 0.046s	sys 0.023s
I20260709 13:20:10.019377 22336 raft_consensus.cc:359] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:10.032171 22336 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:10.038591 22336 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:10.023344 22337 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.050117 22337 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.050761 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:10.051632 16671 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 12: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:10.051930 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.058748 22336 ts_tablet_manager.cc:1434] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.000s
I20260709 13:20:10.071692 17409 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:10.079170 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-d104b6d16beb411c9099f23901cebbb2 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:20:10.079651 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-d104b6d16beb411c9099f23901cebbb2 on tablet d104b6d16beb411c9099f23901cebbb2 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:10.076061 17420 raft_consensus.cc:1217] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Deduplicated request from leader. Original: 14.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 13:20:10.082499 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:10.085546 22375 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:10.086084 22375 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:10.087021 22375 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.069712 22311 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:10.123608 16489 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:10.137730 22343 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.138291 22343 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.143756 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:10.158536 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:10.183564 22400 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:10.183319 22343 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.203009 22400 tablet_bootstrap.cc:492] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:10.203971 22400 ts_tablet_manager.cc:1403] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.287s	user 0.046s	sys 0.004s
I20260709 13:20:10.206012 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:10.206568 17420 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:10.217988 22358 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:10.218580 22358 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.221674 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:10.221956 22358 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.222455 16664 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:10.222827 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:10.223516 17420 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:10.224629 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.225767 22358 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:10.226186 22358 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:10.227492 22358 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:10.235289 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:10.238324 22354 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:10.248234 22354 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.255511 22354 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:10.257905 16489 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 11 to 12, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:10.288611 22400 raft_consensus.cc:359] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:10.290835 22400 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:10.295320 22400 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:10.309770 22400 ts_tablet_manager.cc:1434] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.105s	user 0.000s	sys 0.006s
I20260709 13:20:10.322922 22373 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:10.328596 22373 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.333108 22373 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.334218 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:10.337208 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:10.337901 17409 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:10.340907 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-8b778f817264440987b35333e6540c72 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:20:10.341428 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-8b778f817264440987b35333e6540c72 on tablet 8b778f817264440987b35333e6540c72 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:10.299113 22354 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.343357 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:10.346556 22355 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:10.347025 22355 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:10.347327 22355 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:10.359591 22354 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.361163 22358 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.362263 16736 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:10.367049 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:10.369004 22358 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.369192 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.375905 17409 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:10.377072 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.379750 22208 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:10.380228 22208 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:10.381047 22208 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.381167 22355 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.382443 16664 raft_consensus.cc:2368] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19.
I20260709 13:20:10.385006 22355 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.385272 16708 leader_election.cc:400] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19.
I20260709 13:20:10.387480 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:10.387831 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.388139 16664 raft_consensus.cc:606] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:10.388511 17420 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:10.391042 22366 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.052s	user 0.004s	sys 0.000s
I20260709 13:20:10.391778 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.392448 17415 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:10.391767 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:10.395643 16664 raft_consensus.cc:606] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:10.404224 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:10.399390 17420 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:10.407091 17415 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:10.402043 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:10.408020 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:10.410355 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:10.412528 16587 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:20:10.413080 16587 raft_consensus.cc:606] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:10.414141 22310 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:10.414249 22354 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:10.434326 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:10.434926 16664 raft_consensus.cc:606] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:20:10.432035 22310 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:10.437464 22374 raft_consensus.cc:993] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:10.437523 22310 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.442752 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:10.444998 16664 raft_consensus.cc:606] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:10.451520 22366 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=1 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:10.452482 22366 tablet_bootstrap.cc:492] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:10.453146 22366 ts_tablet_manager.cc:1403] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.724s	user 0.029s	sys 0.017s
I20260709 13:20:10.459239 16489 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 17 to 18, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:10.466951 22358 raft_consensus.cc:1064] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:10.469826 22358 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.474480 22366 raft_consensus.cc:359] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:10.479728 22374 raft_consensus.cc:1081] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:10.482739 16736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:10.483320 16736 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:10.483654 16736 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:10.485704 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:10.487529 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:10.495499 16736 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.501677 16736 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.502586 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.501914 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:10.503188 17409 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:10.503389 16671 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:10.503839 16671 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:10.505733 16671 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.507133 16671 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:10.508683 22433 raft_consensus.cc:993] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:10.509215 22433 raft_consensus.cc:1081] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:10.509774 17409 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:10.510890 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.511662 22208 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:10.497330 22366 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:10.512116 22208 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:10.513450 22208 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.514219 16588 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:10.514449 16736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:10.515239 16736 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:10.517042 16736 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:10.517653 22311 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:10.516789 16671 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:10.500016 22354 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:10.532539 22354 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.535833 16736 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.514444 22366 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:10.546916 22366 ts_tablet_manager.cc:1434] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 13:20:10.547108 16737 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:10.550027 22397 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:10.551120 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-e8b0d179ad714b59b34ab1dc57a57c02 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:20:10.551606 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-e8b0d179ad714b59b34ab1dc57a57c02 on tablet e8b0d179ad714b59b34ab1dc57a57c02 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:10.558274 17415 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:10.569615 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:10.570250 16671 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:10.570521 16671 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:10.571127 16671 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.572120 16671 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:10.611501 17415 raft_consensus.cc:1217] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.8->[16.9-16.9]   Dedup: 16.9->[]
I20260709 13:20:10.615918 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.616513 17415 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:10.605370 16671 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:10.619616 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.622025 16736 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.564963 22431 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:20:10.642445 17409 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:10.653337 17415 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
W20260709 13:20:10.675266 22438 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 13:20:10.680167 22436 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:20:10.680271 22200 raft_consensus.cc:993] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:20:10.681176 22200 raft_consensus.cc:1081] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:20:10.683143 17415 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:20:10.683734 17415 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:10.684063 17415 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:10.687876 22441 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:10.690232 22431 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:10.686555 22440 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:10.690709 22431 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:10.719528 22440 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:20:10.734673 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 88888us
W20260709 13:20:10.752530 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 133010us
I20260709 13:20:10.754738 16671 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:10.743391 17415 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.763262 17415 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:10.768236 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:10.769125 16671 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:10.769543 16671 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:10.734633 22431 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.791240 22200 raft_consensus.cc:993] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:10.791847 22200 raft_consensus.cc:1081] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:10.795418 22431 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:10.777951 22440 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.804768 16671 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.806012 16671 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:10.834115 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59698.9us
I20260709 13:20:10.836907 16664 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:10.854123 16739 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:10.854800 16739 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:10.855468 16739 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:10.856663 16493 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 13 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:10.876286 16671 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:10.881985 22358 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:10.905961 16587 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:20:10.911640 16739 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.919256 16489 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:10.922272 16489 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 18 to 19, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 19 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:10.926147 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:10.926822 16591 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:10.930735 16739 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:10.935937 16591 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:10.941108 16664 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:10.943089 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:10.943681 16664 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:10.943971 16664 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:10.946895 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:10.947611 17420 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:10.948275 16664 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:10.957008 17420 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:10.958274 16710 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:10.958418 17415 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:10.968575 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:20:10.978217 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 110269us
I20260709 13:20:10.979981 16664 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:10.986845 16664 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:10.972687 22375 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:10.971453 22431 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:10.973122 22358 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:20:10.978596 22449 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:20:11.008471 22398 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:11.029032 22398 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:11.036386 22398 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.027757 22337 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:11.045614 22337 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:11.046677 22337 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.048259 22337 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:11.070336 16739 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:20:11.070849 16739 pending_rounds.cc:85] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 11
I20260709 13:20:11.071187 16739 pending_rounds.cc:107] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:20:11.071581 16739 raft_consensus.cc:2889] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:11.073794 22354 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:11.083173 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 79974.6us
I20260709 13:20:11.064006 22375 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:20:11.090687 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 95524.6us
I20260709 13:20:11.096861 22444 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Committing config change with OpId 12.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:11.103595 22445 raft_consensus.cc:993] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:20:11.105561 22445 raft_consensus.cc:1081] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Signalling peer 170921ba08b74a33995ae7c7686f8109 to start an election
I20260709 13:20:11.107079 22375 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.135797 17415 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:11.137290 17409 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:20:11.179571 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 166064us
I20260709 13:20:11.224695 22466 raft_consensus.cc:1064] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:11.239189 22444 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:11.244657 22448 raft_consensus.cc:1064] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:20:11.280275 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 73139.6us
I20260709 13:20:11.311110 17420 raft_consensus.cc:1217] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.8->[15.9-15.9]   Dedup: 15.9->[]
I20260709 13:20:11.315502 16739 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:20:11.316169 16739 pending_rounds.cc:85] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 13
I20260709 13:20:11.316574 16739 pending_rounds.cc:107] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:20:11.262039 22466 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 7, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.302356 22354 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:20:11.313668 22448 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 7, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.317008 16739 raft_consensus.cc:2889] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:11.335749 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52890us
I20260709 13:20:11.344955 17419 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:11.378162 22375 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:20:11.344671 17420 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Committing config change with OpId 12.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:11.434262 17409 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:20:11.438623 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:11.438819 22354 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:11.447351 16668 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:36074
I20260709 13:20:11.463251 16668 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:11.463855 16668 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:11.501751 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:11.502060 22200 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:11.511008 16664 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:11.527860 17413 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:11.533073 16671 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:11.536257 16670 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:11.536573 22471 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:20:11.556162 16668 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.586992 22440 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:20:11.592468 16662 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:11.577844 16665 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Committing config change with OpId 12.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:11.611982 16588 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:11.613838 16589 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:11.667009 16489 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:11.696462 16489 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 14 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:11.730578 22478 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:20:11.780470 16666 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:20:11.696529 22477 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 13:20:11.791415 16668 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:11.832849 16670 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:11.701483 16489 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 15 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:11.868026 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:11.868940 16595 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:11.869378 16595 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:11.870266 16595 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:11.898913 16742 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:11.903198 16595 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:11.923653 16664 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:20:12.013718 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 174764us
I20260709 13:20:12.032347 16595 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:12.037403 17413 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
W20260709 13:20:12.051108 22481 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.277s	user 0.005s	sys 0.000s
W20260709 13:20:12.057595 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 141954us
W20260709 13:20:12.058846 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 174435us
I20260709 13:20:12.077977 22435 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:12.078711 22435 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:12.080077 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 128457us
W20260709 13:20:12.085242 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 112717us
I20260709 13:20:12.091398 22431 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:20:12.070972 22440 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:20:12.093993 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:12.094820 22466 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:12.081326 22435 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:12.105000 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:12.106292 22343 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:12.106969 22343 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:12.108091 22343 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:12.115737 22495 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:12.126827 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43924.9us
W20260709 13:20:12.138192 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61287.4us
I20260709 13:20:12.146657 22229 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:12.164427 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:12.171677 22466 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:12.186676 22311 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:20:12.217555 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 140105us
I20260709 13:20:12.246035 17409 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:12.239888 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:12.258945 17419 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:12.253983 22445 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:20:12.272801 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:12.282084 16595 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Committing config change with OpId 12.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:12.319674 22488 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:20:12.349161 22354 raft_consensus.cc:1064] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:20:12.401146 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 69218.8us
W20260709 13:20:12.407536 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 123461us
W20260709 13:20:12.460598 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 76427.3us
W20260709 13:20:12.637444 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48363.7us
I20260709 13:20:12.666366 17419 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:12.684979 22515 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:20:12.697194 22511 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
W20260709 13:20:12.721638 22513 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:20:12.750023 22509 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 13:20:12.763145 22516 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
I20260709 13:20:12.774539 22518 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:12.775712 22518 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:12.778818 22518 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:12.719296 22354 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 7, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:12.995985 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.019387 16589 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:13.030380 22534 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.031163 22534 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.036620 22535 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:13.037243 22433 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.037231 22535 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.038057 22433 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.040346 22433 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.042428 22433 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.050539 22521 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.042989 22433 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.051146 22521 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.058580 22517 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.059157 22517 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.061260 22517 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:13.061894 22521 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:13.067410 22337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.047s	sys 0.118s
W20260709 13:20:13.067986 22337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.047s	sys 0.118s
I20260709 13:20:13.061820 22517 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.072381 22433 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.077832 22343 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.082599 22343 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.088469 22343 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.098479 22354 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.099085 22354 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.101186 22354 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.102842 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.105304 22535 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.106124 22343 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:13.108057 22519 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:13.108628 22519 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:13.120206 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 75990.9us
I20260709 13:20:13.120944 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.121286 22351 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:13.121537 22466 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.121897 22351 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.122145 22466 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.122581 22519 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.124095 22351 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.124253 22466 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:13.126413 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.133391 22534 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.135735 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.136497 16739 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
W20260709 13:20:13.136783 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.004s	sys 0.008s
W20260709 13:20:13.137096 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.004s	sys 0.008s
W20260709 13:20:13.126754 22528 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.061s	user 0.001s	sys 0.002s
W20260709 13:20:13.139678 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 737796us
I20260709 13:20:13.143884 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:13.144518 22531 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:20:13.144279 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:13.145406 22455 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:13.145797 22455 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.146193 22455 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:13.152068 22539 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 13:20:13.155501 22532 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.382s	user 0.000s	sys 0.002s
I20260709 13:20:13.157397 22455 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.162844 22495 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.163632 22495 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.164346 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.168255 17413 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:13.177702 17420 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:13.191030 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.190166 22200 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:13.178534 22534 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:13.193892 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.110760 17409 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.199960 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.205355 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.217701 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.219910 22526 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:13.220613 22526 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.222334 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.222999 16594 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:13.223127 22455 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.223730 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.223958 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.225050 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:13.225509 22495 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.225633 16739 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:13.226053 22469 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.226650 22469 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.228848 22469 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:13.229409 17416 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:13.230111 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.197067 22488 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:13.231449 22529 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:13.231626 22488 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.206460 22448 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:13.233875 22488 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.233671 22448 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.235359 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.235824 22448 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.239857 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.249468 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.257073 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:13.262992 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46626.6us
I20260709 13:20:13.215363 22474 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:13.283233 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.267922 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.284508 22543 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.285135 22543 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.246023 22526 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.309036 16740 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:13.310403 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.307452 16487 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 15 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.313418 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.333078 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:13.337275 22200 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:20:13.337409 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.283267 22474 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.344719 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.349412 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.329108 16663 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.353464 22489 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:20:13.363360 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.364367 16745 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:13.367067 22525 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:20:13.367662 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:13.369478 22488 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:13.380074 22474 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.391780 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.394447 16745 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 7.
I20260709 13:20:13.395380 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.398380 22489 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:13.401432 16739 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:13.402699 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:13.431135 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.428807 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.433590 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.357707 22543 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:13.444342 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 111950us
I20260709 13:20:13.414741 22489 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:13.448160 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.425424 22535 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:13.449812 16739 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:13.452054 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.453627 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.461536 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.478286 22489 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.479602 22489 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:13.466233 22535 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:13.482254 22535 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.482659 22520 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Committing config change with OpId 14.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.487085 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.488335 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.489184 22440 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:13.489286 22543 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:13.490423 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.491151 22495 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:20:13.492192 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.494474 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:13.495283 22495 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:13.498970 22537 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:13.471483 22550 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:20:13.503537 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 279) took 1925 ms. Trace:
I20260709 13:20:13.511308 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:13.513191 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.513295 22200 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:13.512763 16587 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.521832 22529 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:13.523320 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.525198 16493 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 271) took 1789 ms. Trace:
I20260709 13:20:13.525462 22473 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:13.525899 22473 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.526280 22473 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:13.525672 16493 rpcz_store.cc:276] 0709 13:20:11.735615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.735928 (+   313us) service_pool.cc:224] Handling call
0709 13:20:11.880863 (+144935us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:20:13.524964 (+1644101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26052,"reported_tablets":1,"spinlock_wait_cycles":132645632}
I20260709 13:20:13.527294 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:13.572021 17409 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:13.577637 22551 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.086s	user 0.003s	sys 0.000s
I20260709 13:20:13.505615 16489 rpcz_store.cc:276] 0709 13:20:11.578638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:11.578800 (+   162us) service_pool.cc:224] Handling call
0709 13:20:13.503453 (+1924653us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:12.106939 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:12.107761 (+   822us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:12.107818 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:20:12.107853 (+    35us) tablet.cc:662] Decoding operations
0709 13:20:12.108550 (+   697us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:12.108649 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:12.108679 (+    30us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:20:12.109216 (+   537us) tablet.cc:701] Row locks acquired
0709 13:20:12.109273 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:20:12.109463 (+   190us) write_op.cc:270] Start()
0709 13:20:12.109623 (+   160us) write_op.cc:276] Timestamp: P: 1783603212109394 usec, L: 0
0709 13:20:12.109667 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:20:12.210090 (+100423us) log.cc:844] Serialized 1345 byte log entry
0709 13:20:12.784635 (+574545us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b18003a8640. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7522 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:20:13.067027 (+282392us) op_driver.cc:464] REPLICATION: finished
0709 13:20:13.128763 (+ 61736us) spinlock_profiling.cc:243] Waited 8.72 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327cada0ac 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f328361aef0
0709 13:20:13.228761 (+ 99998us) write_op.cc:301] APPLY: starting
0709 13:20:13.228874 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:13.257569 (+ 28695us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:13.257615 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:13.360623 (+103008us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:13.455458 (+ 94835us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:20:13.455539 (+    81us) write_op.cc:312] APPLY: finished
0709 13:20:13.455881 (+   342us) log.cc:844] Serialized 41 byte log entry
0709 13:20:13.456069 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:13.456162 (+    93us) write_op.cc:454] Released schema lock
0709 13:20:13.456221 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:13.456722 (+   501us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:13.456754 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":9993,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":144137,"apply.run_cpu_time_us":8458,"apply.run_wall_time_us":227937,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16679,"num_ops":3,"prepare.queue_time_us":194479,"prepare.run_cpu_time_us":12845,"prepare.run_wall_time_us":278473,"raft.queue_time_us":302246,"raft.run_cpu_time_us":18886,"raft.run_wall_time_us":140147,"replication_time_us":957171,"spinlock_wait_cycles":32185984,"thread_start_us":433465,"threads_started":4,"wal-append.queue_time_us":174495}]]}
I20260709 13:20:13.598241 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.600291 22526 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:13.603399 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.604470 22523 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:13.608244 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.609619 22517 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:13.610004 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.616559 16663 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.619031 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.641152 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.619658 22526 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:13.651018 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:13.654961 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.663193 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.664214 22530 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:13.666554 22200 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:13.644125 22448 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:13.672423 22525 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:13.643788 22473 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.710907 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 204) took 1287 ms. Trace:
I20260709 13:20:13.711524 16492 rpcz_store.cc:276] 0709 13:20:12.279301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:12.287201 (+  7900us) service_pool.cc:224] Handling call
0709 13:20:12.405077 (+117876us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:20:13.566567 (+1161490us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5687,"reported_tablets":7,"spinlock_wait_cycles":69534080}
W20260709 13:20:13.713198 22556 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:13.719141 16489 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 17, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:13.740743 22542 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:20:13.673962 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:13.693352 22525 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:13.754535 22519 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:13.757866 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 83998.6us
I20260709 13:20:13.753300 22525 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:13.772857 16743 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Committing config change with OpId 14.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:13.776202 22473 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.796847 16666 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:13.811515 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:13.817129 22443 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:13.817682 22443 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.820600 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:13.803112 16492 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:13.819933 22443 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:13.823818 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:13.864696 22474 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:13.865693 22530 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:13.866076 22530 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:13.871946 22525 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.873317 16671 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 7. Current term is 8. Ops: []
I20260709 13:20:13.876952 22525 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:13.877579 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:13.881970 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46460.9us
I20260709 13:20:13.951900 22540 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:13.952600 22540 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:13.923837 22530 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:13.986346 22530 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:13.990602 22540 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:14.006079 16487 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 16, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 16 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:20:14.015184 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 343819us
W20260709 13:20:14.026510 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 94603.2us
W20260709 13:20:14.026821 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 71844.4us
I20260709 13:20:14.105180 16740 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:14.109696 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:14.100216 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:14.039209 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:14.098662 16743 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:14.077688 16739 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:14.036338 22500 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:20:14.116591 22500 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:20:14.116990 22500 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:14.117823 22500 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:14.122241 22500 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:20:14.078243 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50221.7us
I20260709 13:20:14.143203 22498 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:14.159076 17409 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:14.156194 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:14.165598 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:14.175669 22351 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:14.180919 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:14.195487 22373 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Committing config change with OpId 15.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.198805 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:14.216571 16592 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:14.221661 22466 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:20:14.220595 22495 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:14.266655 16742 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.310345 17419 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:14.315495 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48832.1us
I20260709 13:20:14.315539 16487 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.316955 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:14.318244 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:14.320019 22525 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:20:14.326423 22362 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:14.290352 22466 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.334358 16594 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.359841 16587 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:14.359545 17409 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Committing config change with OpId 15.10: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:14.367142 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:14.377384 17419 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:20:14.383767 22495 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:14.384038 16587 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 13:20:14.387333 22522 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:14.406564 22521 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:14.391845 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8b778f817264440987b35333e6540c72 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:14.446517 16594 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3326ab1787714e708abfea6cf0509109"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "170921ba08b74a33995ae7c7686f8109"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:20:14.447294 16594 raft_consensus.cc:606] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Received request to transfer leadership to TS 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:14.447639 16594 raft_consensus.cc:614] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:14.458706 22495 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:14.466336 16589 raft_consensus.cc:2955] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.472927 16594 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:14.523422 17419 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:14.431952 16489 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 9 to 10, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.545550 16745 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:14.546911 16593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:51742
I20260709 13:20:14.549068 16593 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:14.548277 16489 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:14.563114 22466 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Committing config change with OpId 12.13: config changed from index 12 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:14.558883 22575 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:20:14.568393 17416 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Committing config change with OpId 12.13: config changed from index 12 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.582070 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:14.582738 16665 raft_consensus.cc:606] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:14.589751 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:14.590384 16665 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:14.596662 22500 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.599259 22500 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:14.605355 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0ecf0f1df23742a2a06e910e6b7350a0 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:14.606655 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:14.607342 16665 raft_consensus.cc:606] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Received request to transfer leadership to TS ba2df6a668144022a223e504ae571f21
I20260709 13:20:14.613215 16594 raft_consensus.cc:2955] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Committing config change with OpId 12.13: config changed from index 12 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.683525 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:20:14.714476 22578 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.073s	user 0.000s	sys 0.005s
I20260709 13:20:14.718657 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:14.737269 22547 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.105s	user 0.002s	sys 0.004s
W20260709 13:20:14.736533 22528 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.053s	user 0.004s	sys 0.002s
I20260709 13:20:14.763927 22499 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.767715 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b778f817264440987b35333e6540c72 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:59566
I20260709 13:20:14.771685 16745 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.778959 17416 raft_consensus.cc:2955] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:14.799943 16478 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress
I20260709 13:20:14.828058 16487 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 12 to 13, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.834080 22579 tablet_replica.cc:333] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:20:14.835433 22579 raft_consensus.cc:2243] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:14.836346 22579 raft_consensus.cc:2272] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:14.866715 16478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 66914.7us
I20260709 13:20:14.867638 22579 ts_tablet_manager.cc:1916] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:20:14.875010 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52302.5us
I20260709 13:20:14.879895 22373 raft_consensus.cc:993] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:14.883198 22373 raft_consensus.cc:1081] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:14.895612 22488 raft_consensus.cc:993] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:14.896183 22500 raft_consensus.cc:1081] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:14.896430 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d104b6d16beb411c9099f23901cebbb2 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:14.901726 16745 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37636
I20260709 13:20:14.902411 16745 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:14.902787 16745 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:14.910246 22537 raft_consensus.cc:993] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:14.910771 22567 raft_consensus.cc:1081] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:14.919621 16745 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:14.922468 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:14.923095 16593 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:14.923368 16593 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:14.924053 16593 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:14.924348 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:14.924914 16663 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:14.925150 16593 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:14.927977 16487 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 10 to 11, VOTER 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) evicted. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:14.938736 16739 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:14.939338 16739 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:14.939502 16663 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:14.939690 16739 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.940730 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:14.942839 22495 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:14.943050 16745 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:14.943413 22495 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:14.944295 22495 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:14.957661 16593 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:14.966071 16745 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:14.966805 16739 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:14.966683 16745 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:14.973249 16745 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:14.967875 16659 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ecf0f1df23742a2a06e910e6b7350a0 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:59566
I20260709 13:20:14.972328 16660 tablet_service.cc:1558] Processing DeleteTablet for tablet d104b6d16beb411c9099f23901cebbb2 with delete_type TABLET_DATA_TOMBSTONED (TS 170921ba08b74a33995ae7c7686f8109 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:59566
W20260709 13:20:14.986557 16478 catalog_manager.cc:4755] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): ChangeConfig:REMOVE_PEER RPC failed for tablet e8b0d179ad714b59b34ab1dc57a57c02: Remote error: Service unavailable: leader transfer in progress
I20260709 13:20:14.988809 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:14.989379 16663 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:14.989671 16663 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:14.989727 16739 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:14.991284 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:14.987600 16745 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:14.993202 16593 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.990944 16663 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:14.996125 16663 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:14.992976 16487 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 19 to 20, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:14.996304 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:14.998749 16594 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:15.000190 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:15.000716 16665 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:15.000988 16665 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:15.001214 16745 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:15.002274 16665 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:15.006352 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:15.006929 17409 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:15.006958 16665 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:15.008941 16594 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:15.004782 22537 raft_consensus.cc:993] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:20:15.028249 16663 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:15.029434 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:15.030349 22495 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:15.030831 22495 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:15.031661 22495 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.038945 22537 raft_consensus.cc:1081] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:20:15.040019 22579 ts_tablet_manager.cc:1929] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 16.10
I20260709 13:20:15.040954 22579 log.cc:1199] T 8b778f817264440987b35333e6540c72 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/8b778f817264440987b35333e6540c72
I20260709 13:20:15.045965 16665 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:15.050493 17409 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:15.051162 17416 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:15.050000 16593 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:15.048766 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:15.052829 17419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:20:15.053373 17419 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:15.053711 17419 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:15.058564 22466 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:15.060941 22466 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:15.061836 22466 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:15.071240 17419 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:15.073328 17419 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:15.074743 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:15.074851 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8b0d179ad714b59b34ab1dc57a57c02 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 170921ba08b74a33995ae7c7686f8109 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:15.076144 16593 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:15.083390 16593 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:15.084998 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:15.095464 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:15.094815 16665 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:15.096367 22490 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:15.096638 16665 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:15.096966 22490 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:15.097519 16665 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:15.097877 22490 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:15.099004 16665 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:15.093489 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet 8b778f817264440987b35333e6540c72 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:15.096637 16487 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 11 to 12, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.091603 22579 tablet_replica.cc:333] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:20:15.111187 16665 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:15.114989 22579 raft_consensus.cc:2243] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:15.127441 22579 raft_consensus.cc:2272] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:15.117254 22529 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:15.132190 22529 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:15.135599 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:15.142145 22529 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:15.143782 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:15.145632 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:15.146617 22579 ts_tablet_manager.cc:1916] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:15.148134 16636 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:15.147166 16487 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 15 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.149873 22529 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:15.156759 16493 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 18, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 18 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.164863 22488 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:15.165446 22488 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.167624 22488 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:15.168197 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:15.168839 16665 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:15.169118 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:15.169986 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:15.170797 22488 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:15.171253 22488 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:15.171591 22488 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:15.197172 16745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:20:15.197811 16745 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Received request to transfer leadership
I20260709 13:20:15.198278 22488 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.200932 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:15.201468 16665 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:15.201597 16745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:20:15.202207 16745 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Received request to transfer leadership
I20260709 13:20:15.205199 16745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:20:15.205667 16745 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Received request to transfer leadership
I20260709 13:20:15.217473 16665 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
W20260709 13:20:15.218022 16542 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:20:15.218506 16542 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for e8b0d179ad714b59b34ab1dc57a57c02 already in progress
W20260709 13:20:15.218812 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for e8b0d179ad714b59b34ab1dc57a57c02 already in progress
I20260709 13:20:15.218614 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:15.219614 22499 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:15.220218 22499 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:15.221227 22499 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.226689 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:15.234090 22579 ts_tablet_manager.cc:1929] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.12
I20260709 13:20:15.234660 22579 log.cc:1199] T 0ecf0f1df23742a2a06e910e6b7350a0 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/0ecf0f1df23742a2a06e910e6b7350a0
I20260709 13:20:15.236317 22579 tablet_replica.cc:333] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:20:15.237469 22579 raft_consensus.cc:2243] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:15.238452 22579 raft_consensus.cc:2272] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:15.246994 22579 ts_tablet_manager.cc:1916] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:15.249861 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet 0ecf0f1df23742a2a06e910e6b7350a0 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:15.276688 22466 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:15.279021 22466 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.281162 22488 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:15.282219 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:15.283020 16593 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:20:15.284642 22466 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:15.285493 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:15.286103 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:15.286418 16665 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:15.287115 22466 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:20:15.287528 22466 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:15.288951 22466 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:20:15.310338 16487 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:15.316037 22579 ts_tablet_manager.cc:1929] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 14.10
I20260709 13:20:15.316843 22579 log.cc:1199] T d104b6d16beb411c9099f23901cebbb2 P 170921ba08b74a33995ae7c7686f8109: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-1-root/wals/d104b6d16beb411c9099f23901cebbb2
I20260709 13:20:15.325462 16478 catalog_manager.cc:5028] TS 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): tablet d104b6d16beb411c9099f23901cebbb2 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:15.356377 22466 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.378902 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:15.387415 16594 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:20:15.388808 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:15.390363 22440 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:20:15.390916 22440 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:15.391774 22440 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.395831 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:15.396673 16665 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 8: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:15.405687 22466 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 8 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:15.431074 16487 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 7 to 8, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 8 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:15.443610 16665 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:15.450007 22466 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:15.464485 16594 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:15.472810 16594 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:15.473683 22466 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:15.474438 22440 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:15.506932 16663 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:20:15.519765 22495 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:15.520349 22495 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:15.522450 22495 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:15.523837 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:15.528295 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:15.513264 22440 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:20:15.540745 16665 raft_consensus.cc:2368] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:20:15.541913 16708 leader_election.cc:400] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:20:15.542366 16708 leader_election.cc:403] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:15.543344 22440 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:15.547456 16590 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:15.553138 22311 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:15.581734 17409 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:15.587637 22495 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:15.633443 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46941.5us
I20260709 13:20:15.642200 22440 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:20:15.713543 16666 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:15.719830 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:15.736603 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 84602.2us
I20260709 13:20:15.747471 16594 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:15.748708 16666 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:15.767580 22595 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:15.724117 22499 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:20:15.807538 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90399.6us
W20260709 13:20:15.831307 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 65846us
W20260709 13:20:15.889672 22597 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:20:15.900264 22596 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
W20260709 13:20:15.950002 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64903.7us
W20260709 13:20:15.972285 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63148.9us
I20260709 13:20:15.972699 22610 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:20:16.001185 16669 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:16.014693 22609 raft_consensus.cc:993] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:20:16.015367 22595 raft_consensus.cc:1081] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:20:16.024974 16669 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:16.030854 22495 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:20:16.035384 17419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48626
I20260709 13:20:16.036054 17419 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:16.036468 17419 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:16.052727 16741 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:16.056840 22618 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:16.079061 16590 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:20:16.085076 17419 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:16.099216 22499 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:16.119107 22617 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:16.140959 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:16.143164 17419 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:16.142596 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:16.153353 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:16.165390 16741 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:16.165758 16741 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:16.168426 16741 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:16.182770 16741 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:16.186708 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46467.9us
I20260709 13:20:16.188279 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:16.210449 22605 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:20:16.259087 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 112157us
W20260709 13:20:16.315097 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:16.338212 16741 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
W20260709 13:20:16.344153 22627 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:20:16.347777 22624 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 13:20:16.356060 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8b0d179ad714b59b34ab1dc57a57c02 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:16.513036 16591 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:20:16.585698 16561 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 894a0fdf94344f46b0426efbd37a6c40. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:16.587874 16744 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:16.589167 16664 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:16.589213 22638 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:16.590818 22620 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:16.621995 22638 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:16.625782 16664 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:16.651041 16745 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Committing config change with OpId 11.8: config changed from index -1 to 8, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:16.657636 16663 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:16.659113 16489 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index -1 to 8, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 11 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:16.694701 16744 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:20:16.694736 16636 consensus_peers.cc:597] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81853b1e217c4b3aa8033dd7e0fdf211. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:16.697682 16593 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:16.697695 22621 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:16.701786 22621 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:20:16.738806 22621 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:16.740312 16591 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:16.805864 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:17.017095 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:17.174340 16591 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:17.191427 16493 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 10, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:17.192938 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 373758us
I20260709 13:20:17.198491 16744 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3326ab1787714e708abfea6cf0509109"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:20:17.200169 16744 raft_consensus.cc:606] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:17.211751 16745 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:20:17.212527 16745 raft_consensus.cc:606] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:17.213009 16745 raft_consensus.cc:614] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:17.241322 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:17.243573 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:20:17.260186 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 87539.2us
W20260709 13:20:17.279006 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 100335us
I20260709 13:20:17.377107 22665 ts_tablet_manager.cc:933] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
W20260709 13:20:17.414989 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 72462.7us
W20260709 13:20:17.421666 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8b0d179ad714b59b34ab1dc57a57c02 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:17.424264 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 89048.9us
I20260709 13:20:17.429728 22665 tablet_copy_client.cc:323] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:20:17.434635 22666 ts_tablet_manager.cc:933] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:17.434489 16745 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Committing config change with OpId 13.10: config changed from index -1 to 10, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:20:17.448683 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 96399.1us
W20260709 13:20:17.460029 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 119623us
W20260709 13:20:17.460286 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 114185us
I20260709 13:20:17.479849 16665 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:17.480367 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet 894a0fdf94344f46b0426efbd37a6c40 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
I20260709 13:20:17.486104 16739 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:20:17.487505 22537 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:17.488088 22537 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:17.489578 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet 894a0fdf94344f46b0426efbd37a6c40 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-894a0fdf94344f46b0426efbd37a6c40
I20260709 13:20:17.491888 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:17.511071 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:17.511768 16739 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:17.510994 22655 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:20:17.513199 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:17.514559 22653 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:17.515381 22466 raft_consensus.cc:993] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:17.515918 22466 raft_consensus.cc:1081] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:17.519153 22537 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:17.521929 22667 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:17.522486 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:17.523481 17414 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:17.525301 16561 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b40781eb2d304f9b86f5061ae572e1c7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:20:17.530882 22653 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:17.525789 16606 tablet_copy_source_session.cc:215] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:17.536556 22666 tablet_copy_client.cc:323] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:20:17.553802 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 81853b1e217c4b3aa8033dd7e0fdf211 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:20:17.554538 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 81853b1e217c4b3aa8033dd7e0fdf211 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-81853b1e217c4b3aa8033dd7e0fdf211
I20260709 13:20:17.556306 16589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3326ab1787714e708abfea6cf0509109"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38582
I20260709 13:20:17.556936 16589 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:17.557261 16589 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.574076 22665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 894a0fdf94344f46b0426efbd37a6c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:17.588718 22667 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:17.589373 22667 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:17.592463 22667 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:17.593425 22620 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:17.595199 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:17.607256 16745 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:17.599236 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:17.595546 16665 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:17.608347 17414 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:17.609838 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:17.610671 22673 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:17.595335 16589 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:17.619196 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:17.624104 16664 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:17.624786 22673 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:17.625114 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:17.625720 16669 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.631883 22665 tablet_copy_client.cc:806] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:17.632579 22665 tablet_copy_client.cc:670] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:17.633245 16493 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:17.634629 16589 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:17.638387 22665 tablet_copy_client.cc:538] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:20:17.643714 16542 auto_rebalancer.cc:629] Failed to schedule move for tablet 3326ab1787714e708abfea6cf0509109: Remote error: Service unavailable: leader transfer in progress
I20260709 13:20:17.622920 16745 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Committing config change with OpId 15.9: config changed from index -1 to 9, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:20:17.612950 16681 tablet_copy_source_session.cc:215] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:17.651084 16669 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:20:17.652029 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:17.656494 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:17.657123 16739 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:17.657717 16739 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:17.658597 16739 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:17.668217 22673 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:17.669970 16739 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:17.683187 22673 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:17.684118 22673 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:17.686512 16739 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:20:17.691421 22666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81853b1e217c4b3aa8033dd7e0fdf211. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:17.708855 22665 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:17.788363 16493 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 9, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 9 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:17.846165 22666 tablet_copy_client.cc:806] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:17.846902 22666 tablet_copy_client.cc:670] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:17.860139 22666 tablet_copy_client.cc:538] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:17.912742 22637 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:17.913398 22637 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:17.916339 22637 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:17.916850 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:17.917471 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:17.918129 16739 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:17.917513 16665 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:17.920203 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:17.921824 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:17.921984 22637 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:17.923141 22637 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:17.923588 22637 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:17.938360 22665 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:17.922508 16587 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:18.247977 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:18.176508 22637 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:17.969254 22665 tablet_bootstrap.cc:492] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:17.997360 22666 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
W20260709 13:20:19.432849 16587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 1103) took 1515 ms (client timeout 1545 ms). Trace:
I20260709 13:20:19.565490 22595 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.602720 22665 ts_tablet_manager.cc:1403] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 1.912s	user 0.038s	sys 0.005s
I20260709 13:20:19.605345 22637 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:19.628449 22680 ts_tablet_manager.cc:933] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
I20260709 13:20:19.631745 22684 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.633154 22683 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.635058 22681 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.635782 22623 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.723470 22620 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.731547 22638 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:19.813340 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800283ee0 after lost signal to thread 15479
I20260709 13:20:19.731818 22685 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.812455 22698 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.849593 22620 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.849673 22683 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.760775 22616 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:19.764443 22537 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.853416 22620 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:19.762468 22686 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:19.853649 22616 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.853806 22537 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:19.854166 22686 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.854552 22683 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:19.855113 22683 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.855680 22616 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:19.856141 22537 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:19.850589 22685 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.856729 22686 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:19.856817 22537 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.857438 22537 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.858180 22685 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:19.784157 22691 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.782110 22695 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:19.801286 22706 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.778340 22696 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:19.805709 22689 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.806932 22693 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.816071 22694 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:19.816913 22701 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.817552 22682 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.818727 22702 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.804919 22699 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.824138 22697 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:19.825103 16587 rpcz_store.cc:269] 0709 13:20:17.917737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.921597 (+  3860us) service_pool.cc:224] Handling call
0709 13:20:19.432740 (+1511143us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:19.847272 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003cca60 after lost signal to thread 16528
W20260709 13:20:19.940217 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003c84a0 after lost signal to thread 16620
W20260709 13:20:19.941469 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003fcb80 after lost signal to thread 16695
I20260709 13:20:19.848014 22595 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.846321 22684 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.850997 22698 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.856205 22616 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.780699 22692 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:19.785619 22688 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.779551 22709 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.813182 22690 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.928344 22696 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.954661 22616 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.966405 22690 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.967981 22696 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:19.968421 22616 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:19.968779 22690 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:19.968927 22616 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.951525 22710 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:19.969480 22616 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.969743 22710 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.858727 22685 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.855589 22683 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.849452 22687 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:19.942813 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08002777a0 after lost signal to thread 16769
I20260709 13:20:19.858940 22681 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:19.975244 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003b5fe0 after lost signal to thread 17446
W20260709 13:20:19.975683 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4e40 after lost signal to thread 22444
I20260709 13:20:19.933560 22695 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.980958 22695 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:19.849673 22638 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.851060 22623 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.858853 22691 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.986068 22680 tablet_copy_client.cc:323] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.1:34889
I20260709 13:20:19.927433 22706 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.931988 22689 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.937016 22701 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.933023 22693 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.972174 22712 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.935334 22694 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.983245 22705 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:19.949860 22702 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.949945 22699 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.949607 22682 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.951799 22697 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.035224 22707 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.034953 22712 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.035765 22707 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.037427 22712 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.037675 22707 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.038087 22712 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:19.970973 22688 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.981588 22685 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:19.971517 22709 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:19.984068 22687 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.025831 22537 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:19.956542 22692 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.039888 22686 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.038611 22712 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.050062 22690 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.072000 22688 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.071354 22698 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.073287 22620 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.034682 22665 raft_consensus.cc:359] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 11 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.140404 22537 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.147101 22730 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.039794 22705 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.163184 22711 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.188300 22712 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.167840 22665 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 11 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:20.157729 22686 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.161202 22690 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.169250 22721 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.213704 22707 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.214095 22623 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.214785 22720 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.180045 22685 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.215210 22665 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.174801 22537 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.249197 22537 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.203424 22727 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.251868 22595 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.216885 22682 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.190809 22712 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.252812 22712 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.203753 22711 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.256940 22711 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.257673 22733 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.258327 22733 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.259193 22741 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.259730 22741 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.260365 22690 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.261336 22733 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.261751 22741 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.218748 22692 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.262002 22696 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.261936 22733 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.221529 22718 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.220029 22704 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.262773 22696 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.263096 22718 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.263505 22704 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.265079 22696 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.262970 22733 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.222913 22703 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.265671 22704 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.266224 22703 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.222839 22732 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.223385 22735 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.224699 22693 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.225008 22699 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.289222 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:20.289811 16745 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:20.290635 22697 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.292550 22694 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.293730 22687 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.294219 22687 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.294620 22687 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.295152 22740 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.295706 22740 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.184070 22730 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.297601 22637 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.298352 22702 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.238771 22707 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.171638 22715 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.182579 22710 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.240312 22720 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.251720 22727 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.217424 22686 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.219489 22736 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.262300 22731 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.265339 22718 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.222200 22723 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.265628 22696 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.224073 22725 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.223573 22701 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.261775 22690 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.280352 22706 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.281788 22681 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.282401 22616 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.283296 22732 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.284169 22735 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.263015 22724 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.286175 22722 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.288138 22743 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.288730 22713 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.298122 22637 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.300688 22665 ts_tablet_manager.cc:1434] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.466s	user 0.006s	sys 0.011s
I20260709 13:20:20.175678 22620 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.191083 22716 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.320573 16745 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:20.236378 22683 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.230248 22700 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.236963 22721 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.476293 22698 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.691578 22707 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.772399 22707 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.771660 22724 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.687759 22730 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.773555 22730 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.773772 22687 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.738703 16671 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:20.746910 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.775946 22681 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.776429 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.635105 22709 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.702122 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.646071 22623 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.639400 22741 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.778182 22623 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.778301 22741 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.698904 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.696837 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:20.699234 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.780844 22623 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.646499 22734 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.777376 22709 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.781427 22736 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.781803 22734 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.782847 22741 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.685133 22749 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.642937 22729 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.652073 22537 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.640544 22714 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:20.751734 17374 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.545s (SENT) [suppressed 4 similar messages]
I20260709 13:20:20.637029 22738 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:20.703886 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.789247 22714 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.638211 22737 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.484184 22703 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.570324 22733 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:20.639532 22638 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.785795 22721 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.705731 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.636510 22726 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.786190 22722 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.635010 22711 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.415418 22739 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.525264 22693 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.634822 22704 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.564414 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.894855 16742 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
W20260709 13:20:20.774468 22665 ts_tablet_manager.cc:1467] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Tablet startup took 948ms
I20260709 13:20:20.788668 22620 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.789529 22738 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.635800 22717 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.958420 22736 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.959373 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.960132 16668 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:20.782086 22709 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.784739 22537 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.792049 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.565344 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.872004 22684 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.975533 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.783393 22741 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.772042 22698 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.771023 22740 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.698928 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.747851 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.850920 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:20.801847 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.872121 22714 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.874277 22753 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.644196 22695 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.962824 22704 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.784245 22734 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.489571 22702 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.433842 22712 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.791706 22730 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.772936 22707 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.566893 22742 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.962636 22693 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.881331 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:20.881839 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.746258 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.849176 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:20.884822 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.958473 22717 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.998208 22695 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.989259 16737 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:20:21.019583 16592 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:20.791132 22732 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:20.882143 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 130469us
I20260709 13:20:20.956739 22705 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.023273 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.563012 22685 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.471391 22688 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.894589 22711 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.547114 22595 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:21.038947 16743 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:20:20.634579 22697 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.642956 22682 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:20.636042 22694 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.774394 22687 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.377223 22699 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.637393 22751 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:20.781577 22689 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.832285 22716 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.770207 22747 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:20.801273 22638 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:20.900892 22725 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.139427 22638 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.140518 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.141004 22725 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.778622 22637 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.898926 22743 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.986100 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.790792 22737 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:20.988750 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.776471 22681 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.997090 22750 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:21.145449 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.147006 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.147624 17412 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:20.882761 22700 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.997601 22709 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.005028 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.143733 22638 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.149711 16742 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:21.043622 22595 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.953989 16606 tablet_copy_service.cc:140] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Received BeginTabletCopySession request for tablet b40781eb2d304f9b86f5061ae572e1c7 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:47022)
W20260709 13:20:21.020294 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 131849us
I20260709 13:20:21.151644 22700 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.143503 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.151995 16606 tablet_copy_service.cc:161] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Beginning new tablet copy session on tablet b40781eb2d304f9b86f5061ae572e1c7 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:47022: session id = ba2df6a668144022a223e504ae571f21-b40781eb2d304f9b86f5061ae572e1c7
I20260709 13:20:21.153044 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.153204 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.154220 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.154349 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.020819 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.155128 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:21.155939 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.020371 16669 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:20.801642 22749 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.964120 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:20.868614 22690 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.024040 16741 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:21.024667 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.471151 22692 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:21.033224 22758 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:21.035565 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.036371 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.037199 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.037521 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.998739 22537 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.002560 22698 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.041167 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:20.999987 22704 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:20.415783 22715 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.784231 22729 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.985538 22741 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.006454 22691 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.022624 22720 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:20.909616 22713 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:20.893287 22726 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.047274 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.132625 22711 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.133100 22693 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.135653 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.137006 16592 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:21.089229 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.088183 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.144701 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.896250 22731 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.148015 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.144521 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:20.971529 22739 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.147365 22750 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:21.150547 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.868391 22696 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.150697 22681 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.047521 22697 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.070397 22682 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.133859 22722 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.030578 22724 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.156507 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:20.895720 22723 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:20.999121 22753 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.034529 22685 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.034711 22742 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.043272 22688 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.453552 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.455307 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.455619 22688 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.456696 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.458053 22742 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:21.331821 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 174498us
I20260709 13:20:21.458647 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.459589 22713 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.469513 16593 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:21.459761 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.193964 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.459990 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.470969 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.471410 16662 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:21.471913 16742 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:20:21.197033 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.472805 16742 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:21.453336 22595 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.207815 16669 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:21.463883 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.469282 16588 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:21.478853 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.134250 22740 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:21.480108 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.223832 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.235246 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.481276 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.193477 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.397971 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.089207 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.482641 16743 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
W20260709 13:20:21.466041 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 313893us
I20260709 13:20:21.209443 16606 tablet_copy_source_session.cc:215] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:20:21.472261 16740 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:21.258697 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.470721 16738 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:20:21.486930 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.482738 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.484330 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.488019 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.465145 16745 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:21.489435 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.489606 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.490234 16745 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:20:21.490381 16671 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:21.481161 17410 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:21.493355 17419 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:21.493685 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:20:21.493863 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.494216 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.487275 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.495416 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:21.497153 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.495471 16665 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:21.028228 22702 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:21.499390 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.219129 22727 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.156502 22737 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.514509 17376 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:21.500298 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.515760 16592 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:21.516628 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.517180 16592 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:21.356770 22743 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.501542 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.502197 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.221490 22717 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.202459 22749 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.222086 22735 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.148907 22687 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.201460 22751 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.480545 22692 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:21.217658 22716 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.471661 22695 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.497926 22740 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.471783 22730 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.508992 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.510751 22729 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.175666 22747 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.518935 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:21.520926 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet e8b0d179ad714b59b34ab1dc57a57c02 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:21.586885 16480 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 65993us
I20260709 13:20:21.586920 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.587378 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.588111 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.588495 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.588702 16590 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:20:21.589510 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.591264 22747 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.589553 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.596586 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.597380 17413 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:21.598309 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.544404 22682 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.602241 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.603150 16587 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:21.607137 22726 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.590576 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.148739 22699 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:21.621178 22699 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.149035 22694 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.528177 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:21.623353 22694 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.530179 22696 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.625454 22751 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.627210 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.628427 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.530805 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.499696 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.506758 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.531579 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.539223 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.545197 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.537918 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.536247 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.556007 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:21.634796 16588 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:21.557122 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.564591 22702 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.564946 22738 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.565045 22740 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.566354 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.504105 22758 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.568006 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.569859 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.649566 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.650022 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.571990 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.650754 16593 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:21.572199 22741 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.541390 22739 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.574179 22687 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.574967 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.575577 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.657311 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.658504 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.576228 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.651093 22729 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:21.659578 22729 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:21.660010 22729 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:21.649209 22696 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:21.660809 17409 raft_consensus.cc:1217] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 11 LEARNER]: Deduplicated request from leader. Original: 11.7->[11.8-11.8]   Dedup: 11.8->[]
I20260709 13:20:21.661023 22696 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:21.576730 17411 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:21.661545 22696 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:21.580184 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.583909 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:21.666265 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.667347 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:20.898708 22665 ts_tablet_manager.cc:1469] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Trace:
0709 13:20:17.268746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.270959 (+  2213us) service_pool.cc:224] Handling call
0709 13:20:17.379610 (+108651us) ts_tablet_manager.cc:934] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
0709 13:20:17.572634 (+193024us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:17.593281 (+ 20647us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:17.593410 (+   129us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:17.630406 (+ 36996us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:17.630688 (+   282us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:17.631001 (+   313us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:17.631056 (+    55us) inbound_call.cc:177] Queueing success response
0709 13:20:17.690232 (+ 59176us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:20:19.843879 (+2153647us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:20:20.032144 (+188265us) tablet_replica.cc:261] Starting instrumentation
0709 13:20:20.032817 (+   673us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":250,"mutex_wait_us":29570,"raft.queue_time_us":28809,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"tablet-copy.queue_time_us":99985,"tablet-download-pool.queue_time_us":114,"tablet-download-pool.run_cpu_time_us":1118,"tablet-download-pool.run_wall_time_us":3902,"thread_start_us":115199,"threads_started":2}
I20260709 13:20:21.671054 22731 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.544032 22681 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:20:21.584697 22690 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.585530 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.586112 22697 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.618137 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.523765 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.627382 22692 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.631362 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.534667 17410 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:21.535792 22743 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:21.638233 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 110128us
W20260709 13:20:21.644048 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 88121.9us
I20260709 13:20:21.678686 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.567179 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1400) took 1280 ms. Trace:
I20260709 13:20:21.680225 16739 rpcz_store.cc:276] 0709 13:20:20.286519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:20.290957 (+  4438us) service_pool.cc:224] Handling call
0709 13:20:20.291280 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:21.493616 (+1202336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:21.493643 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:21.493680 (+    37us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:21.493714 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:21.566959 (+ 73245us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400278dd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:21.567000 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:21.567117 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2764739968}
I20260709 13:20:21.647348 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.675740 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.648875 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.558809 22707 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.675747 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.648562 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.678722 17416 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:21.693405 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.925601 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.927033 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.927667 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.928792 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.927716 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.935420 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:21.704596 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 216044us
I20260709 13:20:21.736848 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.940312 16743 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:21.940819 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 460626us
I20260709 13:20:21.734212 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:21.945219 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 439143us
I20260709 13:20:21.773804 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.655587 22685 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:21.771764 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 168604us
I20260709 13:20:21.953138 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:21.695072 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:21.862192 22666 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:21.959427 22690 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:21.960889 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.961400 22666 tablet_bootstrap.cc:492] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:21.962034 22743 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:21.962723 22743 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:21.938161 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:21.965394 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:21.967438 22688 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:21.837251 22696 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.968076 22688 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:21.968663 22688 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:21.969506 17410 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:21.971096 22666 ts_tablet_manager.cc:1403] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 3.975s	user 0.058s	sys 0.028s
I20260709 13:20:21.789914 22750 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.791649 22753 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:21.982733 22764 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:21.984148 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:21.983839 22764 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.986358 22707 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:20:21.986850 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 43993.2us
I20260709 13:20:21.987362 22707 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:21.987928 22707 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:21.987829 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:21.988890 22764 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.772459 22715 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:21.792711 22758 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.990383 22763 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:21.809861 22729 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:21.991573 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:21.991487 22763 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:21.992627 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:21.772759 22723 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:21.968099 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:21.998878 16590 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:21.999790 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.001869 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.002389 22729 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.003865 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.004601 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.014678 22685 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:22.015168 22685 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:22.015501 22763 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.015630 22685 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:22.016265 22763 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.017685 16710 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.019228 22750 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:22.020289 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46935.8us
I20260709 13:20:22.021375 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.023600 22762 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.024134 22762 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:22.024525 22762 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.028653 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.030030 22691 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:22.030536 22691 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:22.031054 22691 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.031632 22761 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:22.032141 22761 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:22.032723 22761 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.035506 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.036828 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.038709 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:22.032063 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.041209 22764 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:22.042225 22764 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.046842 22692 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:22.047319 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.047822 22764 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.048161 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.047420 22692 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:22.048834 22692 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:22.048962 22696 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.049706 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.050925 22729 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:22.051312 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.051397 22729 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:22.044631 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 109623us
I20260709 13:20:22.057235 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.058199 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.058928 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.053892 22729 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:22.030526 22685 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.063541 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.064962 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.068014 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.074853 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.075353 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.069406 22690 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:22.073843 22537 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:22.077502 22537 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.077669 22690 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:22.082652 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 80948us
I20260709 13:20:22.079054 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.076738 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.077967 22537 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.086761 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.078739 22690 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.097577 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.097891 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.099478 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.100419 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.100536 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.101606 16664 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:22.102154 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:22.107676 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75728us
W20260709 13:20:22.107566 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 43550.7us
I20260709 13:20:22.110831 22699 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:22.116626 22699 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:22.117079 22699 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.118320 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.119330 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.123431 22762 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.126173 22762 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.128712 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.131141 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.135640 22743 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.084015 22666 raft_consensus.cc:359] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:22.144563 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.146670 22685 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.147398 22685 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.147959 22685 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.148394 22685 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.142093 22666 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:22.157648 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.157938 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.165712 22691 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.180785 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:22.180706 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.187824 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.191648 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.192857 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.199942 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.200704 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.203764 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.205165 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.208572 22688 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.184688 22666 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:22.217026 22666 ts_tablet_manager.cc:1434] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.245s	user 0.002s	sys 0.007s
I20260709 13:20:22.218075 22692 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.219127 22688 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.219791 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.220443 22692 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:22.223264 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 78825.6us
I20260709 13:20:22.222815 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.203265 22691 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.228938 22764 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:22.229686 22764 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.231727 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.232076 22764 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.223012 22699 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.226565 22690 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.241110 22699 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.241519 22690 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.236150 22537 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:22.242933 16664 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:22.243659 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.244445 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.246534 16668 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:22.249583 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.228005 22707 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.251746 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.253330 22766 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:22.253731 22766 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.253959 16668 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:22.254119 22766 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:22.254949 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.256834 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.225522 22761 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.258989 16670 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:22.259618 22537 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.254105 22707 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.263732 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.265023 22715 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:22.265595 22715 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:22.265671 22766 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.266220 22715 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.268148 22766 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.280462 22729 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.283542 22729 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.287572 22680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40781eb2d304f9b86f5061ae572e1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:20:22.289240 22761 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.291525 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.292397 16592 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:22.300607 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.302033 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.303066 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.304296 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.304682 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.305792 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.308068 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:22.309832 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.312320 22715 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.314546 22715 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.317080 16669 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:20:22.319698 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.319528 22685 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.320999 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.321614 22743 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.322026 22685 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.323504 22729 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:22.327698 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.321779 17414 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:22.325206 22681 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:22.330603 22681 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:22.330986 22681 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.332062 22681 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.333401 22681 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.323882 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.328729 22727 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:22.334972 22723 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.335372 22727 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.335450 22723 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.335915 22723 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.337030 22537 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:22.337441 22537 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.337823 22537 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.338650 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:22.339248 16668 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.346495 16668 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:22.336550 22727 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:22.351352 22727 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.351222 22723 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:20:22.352366 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44670.8us
I20260709 13:20:22.353565 22723 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.357477 22537 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.358594 16664 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:22.361304 22537 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.361490 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.362318 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.362435 22695 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:22.363209 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 106514us
I20260709 13:20:22.363590 22695 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.364729 22695 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.368440 22694 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:22.370144 22694 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:22.370406 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.370586 22694 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:20:22.371199 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 80258.3us
I20260709 13:20:22.372201 22694 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.373492 22694 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.377069 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.397636 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.403326 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.406471 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.406204 22762 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.410599 22762 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.410988 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.411183 16493 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 13, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:22.412328 22685 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:22.415815 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:22.422758 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 185792us
I20260709 13:20:22.423933 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.426225 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.427657 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.430040 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.430650 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.430795 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.431326 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.432104 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.435204 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.442615 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.444032 16664 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:22.445490 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.445604 22715 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:22.446583 22715 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:22.446991 22715 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.447541 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.448050 22715 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.449296 22715 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:22.449949 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:22.454227 16590 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:22.457113 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.459338 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.461546 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.465807 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:22.466391 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.466300 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.468972 22758 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.472976 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.473683 16634 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:22.475369 22537 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:22.475907 22537 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:22.475963 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.477304 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:22.477566 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.483873 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.485003 16736 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:22.485133 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.486362 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.486683 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.487407 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.488415 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.488806 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.489666 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.490093 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:22.491380 22537 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:22.500105 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 74667.7us
W20260709 13:20:22.500981 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.506527 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.507251 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.507781 22715 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:22.508517 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.508646 22690 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.508882 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.510772 22715 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.512550 22694 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:22.513762 22715 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:22.514138 22715 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.517131 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:22.521064 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:22.521512 22764 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.521821 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:22.521625 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.522910 22681 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.529153 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.531181 22764 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:22.516795 22715 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.533144 22715 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.540449 16664 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:22.544492 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.545393 22723 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.545809 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.547461 22723 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.547914 22723 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.548324 22723 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.552913 16664 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:22.542294 22694 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:20:22.554935 22681 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:22.555723 22680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:48636 (request call id 884) took 4612 ms. Trace:
I20260709 13:20:22.555845 22681 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:22.556581 22681 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.556537 22680 rpcz_store.cc:276] 0709 13:20:17.943911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:17.944018 (+   107us) service_pool.cc:224] Handling call
0709 13:20:19.845060 (+1901042us) ts_tablet_manager.cc:934] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889)
0709 13:20:22.285629 (+2440569us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:20:22.298593 (+ 12964us) tablet_metadata.cc:665] Metadata flushed
0709 13:20:22.298714 (+   121us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:20:22.470420 (+171706us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:20:22.470920 (+   500us) tablet_replica.cc:206] Creating consensus instance
0709 13:20:22.520466 (+ 49546us) ts_tablet_manager.cc:989] Replica registered.
0709 13:20:22.555590 (+ 35124us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":251,"tablet-copy.queue_time_us":1683836,"thread_start_us":180808,"threads_started":2}
I20260709 13:20:22.557663 22681 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.559003 22681 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.559006 22680 tablet_copy_client.cc:806] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:20:22.561229 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.562474 22695 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.561718 22680 tablet_copy_client.cc:670] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:20:22.555930 22694 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:22.567440 22694 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.568557 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.569542 22766 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.568814 22694 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:22.571996 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.572206 22694 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.572530 22723 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.573513 22766 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:22.575249 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.575677 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.562901 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.580142 22766 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.580425 22695 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.581593 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.581290 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.583483 22695 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.583540 22766 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.583925 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.584122 22766 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:22.584523 22766 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.584780 22695 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.584456 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.593629 16736 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 10.
I20260709 13:20:22.586617 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.595106 22707 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:22.614508 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.615645 22764 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.616590 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.619141 22723 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.629729 16487 catalog_manager.cc:5697] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 11 to 12, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:22.651432 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 69915.8us
I20260709 13:20:22.650691 22766 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:22.653308 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 66788.5us
I20260709 13:20:22.661381 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.662281 22695 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:22.662465 22766 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.653874 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.658748 22681 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.665297 22723 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.666209 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.667131 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.667634 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:20:22.667160 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50641.3us
I20260709 13:20:22.667208 22707 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.668736 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.669498 22723 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:22.669884 22723 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:22.669984 22681 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:22.670337 22723 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:22.671249 22695 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.671754 22695 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:22.672178 22695 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.677903 22723 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.679642 22707 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.680593 22695 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.680963 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.683792 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:22.686053 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.687898 22707 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.688727 22537 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:22.688848 22695 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:22.689276 22537 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:22.689739 22537 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.691607 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.692548 22695 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:22.693022 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.693815 22695 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.696223 22723 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.697453 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.699499 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.706816 22707 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.694525 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.715660 22695 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.716197 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.717097 22695 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.717440 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.717442 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.718984 22695 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:22.720886 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.698300 22537 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.724921 16561 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.725795 22694 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.727166 22723 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.727981 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.726356 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.729050 22707 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.731954 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.732914 22694 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.734573 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.736212 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.733953 22723 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.737133 22685 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.737685 22723 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:22.738472 22766 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.723480 22537 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.738513 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.740238 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.741140 22537 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.742278 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.747642 16664 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.748092 22694 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:22.748899 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.749781 22694 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.750278 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.750581 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.750639 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.751206 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.742730 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.752269 22707 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.753891 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.754660 16664 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:22.755107 22694 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.756686 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.757954 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.759696 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.762894 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.761225 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.763717 22707 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.763785 22694 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:22.764817 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.765156 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.765156 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.766093 22707 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.766630 22694 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.766760 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.767612 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.768457 22707 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.768880 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.770401 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.770694 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.771209 22707 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.772591 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.772789 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.773375 16670 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:22.773516 22707 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:22.773909 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.774511 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.776587 22707 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:22.777710 22764 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:22.778119 22764 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:22.779060 22764 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.779748 22695 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:22.780813 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-81853b1e217c4b3aa8033dd7e0fdf211 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:20:22.780833 22695 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:22.781544 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-81853b1e217c4b3aa8033dd7e0fdf211 on tablet 81853b1e217c4b3aa8033dd7e0fdf211 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.781487 22695 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:22.782562 22695 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.779458 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:22.768157 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.783614 16668 raft_consensus.cc:2376] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18. Re-sending same reply.
I20260709 13:20:22.784346 16667 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:22.785845 22695 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:22.791191 22681 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:22.791966 22764 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:22.792584 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.793390 22681 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.794555 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.805799 22681 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:22.807260 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.809906 22681 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.807157 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.811851 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.812223 22707 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.815977 22695 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:22.816461 22707 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:22.817931 22763 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:22.818328 22763 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:22.819319 22763 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.819918 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.819972 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.821024 16668 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:22.823375 22763 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:22.823650 16670 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.823998 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.825493 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.829141 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.829514 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.830348 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.831738 22695 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:22.832386 22695 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.832981 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.833796 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.834448 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:22.835350 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.835464 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.837092 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.837905 16670 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:22.837285 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.840794 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.843703 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.843837 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.845058 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.845126 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.846424 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.847326 22758 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:22.848543 22758 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:22.848841 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.849509 22758 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:22.849885 22758 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:22.850189 22753 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:22.850757 22758 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.852006 22758 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:22.852468 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:22.838781 16594 raft_consensus.cc:2376] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18. Re-sending same reply.
I20260709 13:20:22.859485 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.860179 22537 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.859848 22695 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:22.860603 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.868177 16670 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:22.868168 22537 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:22.868209 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.869696 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.870579 22766 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.870826 22537 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.871320 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:22.872233 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.872885 16668 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:22.872968 22537 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:22.873375 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.851857 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.874284 22537 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:22.874289 22758 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.875129 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.874969 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.875954 22758 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.876047 22537 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:22.876435 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.877272 22724 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.877360 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.878302 22724 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.880260 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.883451 22537 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:22.883570 22758 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.884641 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.886780 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.885636 17374 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.886183 22537 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:22.888792 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-894a0fdf94344f46b0426efbd37a6c40 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:20:22.889432 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-894a0fdf94344f46b0426efbd37a6c40 on tablet 894a0fdf94344f46b0426efbd37a6c40 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.890977 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.891695 16587 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:22.892637 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.888695 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:22.895201 22753 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.897174 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.897281 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:22.897825 16590 raft_consensus.cc:2376] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18. Re-sending same reply.
I20260709 13:20:22.900585 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.901503 16591 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:22.901618 22753 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.905928 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.911512 22753 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.912602 22537 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.913125 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.914001 22537 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.915807 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.917042 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.918269 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.919121 22537 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:22.919624 22537 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:22.920255 22753 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.920519 22537 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.921185 22753 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.922734 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:22.924651 16587 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.927018 22758 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:22.928284 22758 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:22.929304 22758 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.934618 22753 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:22.937153 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.938128 22724 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:22.938537 22724 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:22.939057 16668 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:22.939509 22724 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.939564 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:22.940778 22724 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:22.940847 22707 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:22.941402 22764 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:22.942014 22707 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:22.943609 22707 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:22.944693 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:22.944525 22707 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:22.950223 22753 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:22.950362 22707 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:22.951807 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.946509 16590 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:22.942133 22690 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 13:20:22.964378 22685 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:22.965118 22724 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:22.969247 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:22.971455 22753 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:22.979676 22753 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:22.980049 22753 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:22.980995 22753 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.982216 22753 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:22.964329 22707 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:22.985538 16739 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:22.985843 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:22.976312 16487 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 9 to 10, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 10 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:22.984992 22707 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:22.987536 22707 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:22.995041 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:22.999185 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57459us
I20260709 13:20:23.000777 22707 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:23.002512 22690 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:23.004285 22777 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:23.004669 22777 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:23.008466 22777 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.008507 22685 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:23.011484 22758 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:23.012801 22758 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:23.014608 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.013197 22777 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:23.015781 22763 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:23.014662 16487 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 16 to 18, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 18 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:23.013195 22758 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:23.019829 22758 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.021165 22758 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.022028 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:23.022646 16743 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
W20260709 13:20:23.027071 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57902.4us
I20260709 13:20:23.027912 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:20:23.032972 22776 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:20:23.010345 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.043045 22537 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:23.044006 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.046458 22537 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:23.053179 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.055104 22537 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:23.055702 16743 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:23.045567 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.057046 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.058279 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.059144 16594 raft_consensus.cc:2410] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:20:23.060243 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.060354 22782 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:23.061669 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.063048 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.061906 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.064429 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.065009 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.064414 22753 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:23.067171 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.068557 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:23.069862 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:23.070508 16587 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:23.072135 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.072995 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:23.074921 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.081626 16743 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:23.086476 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.088323 16587 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:23.088605 22753 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:23.090941 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.091691 22537 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:23.095028 22753 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:23.096516 17374 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.097277 22782 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:23.103017 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.104031 22777 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:23.105126 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.105427 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.106338 22777 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:20:23.107039 22758 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:23.107867 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.111505 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.112632 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.113354 22763 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:23.114053 22763 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:23.114653 22763 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:23.116662 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.117120 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.117961 22763 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:23.119277 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.120514 22777 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:23.121639 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.122781 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.124076 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.109843 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:23.125272 22758 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:23.125720 22695 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:23.125679 22758 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:23.126691 22758 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:23.126767 17374 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.127635 22782 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:23.135053 22763 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:23.134824 22695 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:23.135792 22763 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:23.136089 22681 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:23.136482 22763 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:23.136483 22681 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:23.138196 22681 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.139997 22758 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.141917 22758 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:23.147205 22681 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.153793 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:23.128001 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.157444 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.160292 22707 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:23.164873 22707 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:23.165920 22777 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:23.166353 22777 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:23.166077 22782 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:23.167201 22777 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:23.169950 22777 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:23.172498 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.173836 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:23.174620 22695 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:23.175365 22695 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:23.176602 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:23.177642 22695 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:23.178037 22695 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.178431 22695 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:23.178891 22782 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:23.179445 22782 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:23.180332 22782 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.189435 22695 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.192829 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:23.193436 16668 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:23.203812 22695 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:23.205901 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:23.206684 16739 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:23.213897 16739 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:23.221688 16668 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:23.223453 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:23.224296 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:23.224979 22695 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:23.225553 22695 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:23.226534 22695 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.234107 22680 tablet_copy_client.cc:538] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:20:23.252583 22680 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:20:23.257057 16590 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:23.259548 22685 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:23.260843 22685 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:23.263453 22685 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:23.282375 22685 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:23.283872 22685 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:23.266041 16493 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 18 to 19, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 19 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:23.313474 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:23.330241 17411 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:23.331481 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:23.336032 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:23.347764 16739 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:23.349444 16667 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:23.349392 22758 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:23.350418 22758 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:23.350757 22758 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:23.351889 22758 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.353659 22758 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:23.350750 17416 raft_consensus.cc:1217] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Deduplicated request from leader. Original: 13.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:20:23.356528 22681 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:23.357491 22681 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:23.357757 22681 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:23.358544 16590 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:20:23.358997 16590 pending_rounds.cc:85] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 6
I20260709 13:20:23.358500 22681 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.359855 22681 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:23.359210 16590 pending_rounds.cc:107] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:20:23.362090 16590 raft_consensus.cc:2889] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:23.363523 22786 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:23.369710 17409 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:23.371673 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:23.428174 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:23.470323 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 71263.9us
I20260709 13:20:23.478564 16739 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:20:23.477834 16493 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 16, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 16 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:23.481717 22723 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:20:23.494004 16594 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:23.509298 16668 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:23.510607 22788 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:23.547015 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 108057us
I20260709 13:20:23.548342 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:23.550336 17409 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:20:23.551569 22758 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:23.552155 22758 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:23.553062 22758 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.573168 22695 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:23.608144 16739 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:23.608479 22692 raft_consensus.cc:1064] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:23.609830 22758 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:23.616010 22692 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:23.630323 16487 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 14 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 15 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:23.664072 16742 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:23.666011 22793 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:23.718179 16590 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:23.720073 22791 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:23.722081 16743 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:23.723695 17414 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:23.725708 22685 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:23.728344 22537 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:23.736842 16594 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:23.741907 22782 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:23.744834 16668 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:23.746517 22793 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:23.824537 17420 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:20:23.831875 17420 pending_rounds.cc:85] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 11
I20260709 13:20:23.832134 17420 pending_rounds.cc:107] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:20:23.832404 17420 raft_consensus.cc:2889] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:23.821339 22809 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:20:23.846935 16742 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:23.846963 22789 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:20:23.839615 22692 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Committing config change with OpId 13.11: config changed from index 10 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:23.915195 22788 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.915753 22788 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:23.918013 22788 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:23.919122 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:23.919770 16736 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:20:23.921391 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:23.921518 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:23.922183 16668 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:23.922308 22789 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:23.922797 22789 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.923169 22789 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.933753 17414 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:23.936455 22789 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:23.939453 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:23.940016 16668 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:23.941756 22707 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.942292 22707 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:23.944923 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:23.946192 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:23.946691 22707 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:23.946808 16667 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:23.946905 16668 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:23.948127 16559 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:23.948294 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:23.948851 22788 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:23.949339 22788 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:23.949358 22707 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:23.949913 22707 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:23.950237 22707 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:23.950523 22789 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:23.951993 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:23.952525 16742 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:20:23.951822 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46775.5us
I20260709 13:20:23.950172 22788 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:23.963624 16594 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Committing config change with OpId 13.11: config changed from index 10 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:23.963590 16489 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 10 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:23.973397 16742 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:23.978188 22777 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:23.978814 22777 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:23.993332 22777 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:23.994925 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:23.995664 16668 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
W20260709 13:20:23.997080 22680 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.066s	user 0.003s	sys 0.001s
I20260709 13:20:23.998535 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.003412 22764 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:24.003868 22764 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.002899 16487 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.004218 22764 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:24.007663 22788 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.008168 22788 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.010334 22788 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.012156 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.012917 16668 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:24.014245 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.015883 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:24.016407 22680 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=2 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:20:24.016501 16742 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:24.017035 22789 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:24.017735 22789 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.018085 22789 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:24.011073 22707 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.024355 22707 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.025760 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:24.036011 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.037878 16668 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.047541 22680 tablet_bootstrap.cc:492] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:20:24.048396 22680 ts_tablet_manager.cc:1403] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.796s	user 0.054s	sys 0.026s
I20260709 13:20:24.049191 16594 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
W20260709 13:20:24.052858 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54399.7us
I20260709 13:20:24.056912 17414 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:24.058398 22537 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:24.059705 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:24.068521 16742 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:20:24.070333 22537 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:24.071960 22680 raft_consensus.cc:359] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:24.072840 22680 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:20:24.073555 22680 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:24.074735 16591 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:20:24.076210 22758 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:24.077706 22789 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.082980 22789 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.083407 22680 ts_tablet_manager.cc:1434] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.001s
I20260709 13:20:24.083772 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.084378 16667 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:24.087658 22786 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:24.100550 16670 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:20:24.102152 17415 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:24.106448 22764 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.108541 22764 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.109759 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.110347 16668 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:24.110409 16606 tablet_copy_service.cc:342] P 3c3ca2a0390b46bb9a5eda03a64eda1a: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-b40781eb2d304f9b86f5061ae572e1c7 received from {username='slave'} at 127.0.0.1:47022
I20260709 13:20:24.110968 16606 tablet_copy_service.cc:434] P 3c3ca2a0390b46bb9a5eda03a64eda1a: ending tablet copy session ba2df6a668144022a223e504ae571f21-b40781eb2d304f9b86f5061ae572e1c7 on tablet b40781eb2d304f9b86f5061ae572e1c7 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:20:24.109990 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:24.112341 16590 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:20:24.114707 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:24.115307 16736 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:24.119371 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.120128 16670 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:24.129426 22758 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:24.135859 16590 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:24.137991 16736 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:24.140209 16667 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:24.142481 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.142350 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.149736 22788 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:24.151492 22788 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:24.152505 22707 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:24.151813 16743 raft_consensus.cc:2955] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Committing config change with OpId 13.12: config changed from index 11 to 12, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:24.152375 22788 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.153201 22707 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:24.154945 22813 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.155457 22813 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.157538 22813 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.158990 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.159605 17414 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:24.160689 16710 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.161499 22764 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:24.161856 22764 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.162184 22764 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.154258 22707 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.169663 16668 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:24.170441 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.171164 16669 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:24.171191 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:24.172043 22813 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:24.172696 22813 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:24.173489 22813 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.197958 16670 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:24.203944 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81853b1e217c4b3aa8033dd7e0fdf211 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:20:24.187927 22777 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.211308 22681 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.211694 22777 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.217731 16491 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 11 to 12, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:24.223572 16489 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:24.211872 22681 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:24.227474 22681 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.227692 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.228502 16670 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:24.229614 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.230540 22681 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:24.231012 22681 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.231418 22681 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.231704 22777 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.237861 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.235570 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:24.239152 16739 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:24.240816 16670 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:24.240517 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.241837 17415 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:24.243613 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.247781 22818 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:24.248364 22818 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.248807 22818 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.292959 22764 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.297712 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.298305 16670 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.311267 16670 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:24.312736 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.323119 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:24.323750 17415 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.321903 22777 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:24.329635 22777 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:24.330566 22777 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.331048 16585 tablet_service.cc:1558] Processing DeleteTablet for tablet 81853b1e217c4b3aa8033dd7e0fdf211 with delete_type TABLET_DATA_TOMBSTONED (TS 3c3ca2a0390b46bb9a5eda03a64eda1a not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:42550
W20260709 13:20:24.331600 22816 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:20:24.348430 22814 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:20:24.339314 22758 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.354040 22758 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.325073 22764 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.362005 16739 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:24.363202 22813 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.363751 22813 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.365794 22813 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.365971 22818 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.338526 17415 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:24.388001 22793 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.395169 22791 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:24.399732 22791 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.402012 22791 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.386520 22764 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:24.405695 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.411612 16594 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:24.407199 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:24.412597 16743 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:24.387266 22789 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:20:24.415057 16492 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.404393 22681 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:24.425288 22681 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.392494 22793 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.427016 22764 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.429888 22764 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.435945 22758 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.436621 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:24.437626 16664 raft_consensus.cc:606] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:24.450255 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.450947 16664 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:24.452495 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 139849us
I20260709 13:20:24.404225 22695 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:24.458093 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.458767 16670 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:20:24.461427 22818 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.462095 22695 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.463413 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:24.464032 17415 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.463872 16492 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:24.464522 22695 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.467078 22793 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.470554 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:24.471683 22792 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.473202 17415 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:24.471319 22813 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:24.493454 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.478922 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.479722 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.480808 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.490814 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.494922 16669 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.494972 16668 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:24.491050 22822 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:24.495808 22822 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:24.496196 22822 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:24.498824 22792 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.500727 16667 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.500931 22792 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.502466 16664 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:24.503141 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.503513 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.504213 16594 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:24.506274 16670 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.507069 16669 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:24.508014 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.506300 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.506263 22826 tablet_replica.cc:333] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:20:24.506363 22813 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.510758 22707 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:24.511366 22707 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:24.511618 22813 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.512246 22707 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.512770 22792 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:24.513192 22792 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.513549 22792 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.513751 22788 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:24.514358 22788 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:24.516479 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:24.517166 16669 raft_consensus.cc:606] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:24.515231 22788 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:24.531744 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.513531 22826 raft_consensus.cc:2243] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:24.515964 22695 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:24.536782 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:24.538935 16743 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.541548 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:24.542173 16742 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:24.545020 22826 raft_consensus.cc:2272] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:20:24.554018 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46087us
I20260709 13:20:24.538793 22695 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.559862 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.561085 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.561985 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.566793 16590 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:24.560881 22793 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:24.564002 16743 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:24.568768 22793 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.566056 16587 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:20:24.566093 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.566542 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:24.572173 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.554018 22685 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.572793 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:24.567832 22695 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.558995 22692 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.561549 22827 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:24.597746 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.600915 16670 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:24.607908 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.610105 17414 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:24.600869 22829 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:24.610846 22793 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.621955 22826 ts_tablet_manager.cc:1916] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:24.607604 22758 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.588486 22685 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.628021 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:24.605581 22822 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.602172 22692 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.638808 16667 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:24.623489 22829 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.642136 22764 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:24.644493 16667 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:24.647626 22792 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.639549 16492 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.608532 22827 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.654932 16669 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.641644 22829 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:24.631644 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64543.1us
I20260709 13:20:24.638571 22820 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:24.686384 17414 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 11 LEARNER]: Advancing to term 13
I20260709 13:20:24.670296 16667 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:24.672688 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.687742 17420 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:24.632866 22758 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.698169 22822 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.703761 16664 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.704276 22792 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.696012 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:20:24.719717 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 62716.3us
I20260709 13:20:24.728962 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:24.729403 22823 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.729635 16587 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.729984 22823 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.732177 22823 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.686384 22820 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:24.737988 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:24.742610 16593 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:24.746055 17414 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:24.750365 22788 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:24.751498 22824 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:24.751964 22824 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.752332 22824 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.741648 22820 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:24.688786 22834 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:20:24.767350 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.768072 16593 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:20:24.764372 22758 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.779074 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.781319 22822 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:24.781718 22822 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.782070 22822 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.787140 22692 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.800292 22827 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.801457 22695 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.802449 22685 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.802662 16591 raft_consensus.cc:2368] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:24.803704 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.804373 16593 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:24.807191 16634 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.808173 22786 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:24.808622 22839 raft_consensus.cc:993] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:24.808616 22786 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:24.809270 22839 raft_consensus.cc:1081] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:24.809342 22786 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:24.816545 16592 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:20:24.817476 16592 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:24.817993 16592 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:24.803733 22695 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.825227 22826 ts_tablet_manager.cc:1929] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 13.11
I20260709 13:20:24.825829 22826 log.cc:1199] T 81853b1e217c4b3aa8033dd7e0fdf211 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-0-root/wals/81853b1e217c4b3aa8033dd7e0fdf211
I20260709 13:20:24.828020 22829 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.828683 22824 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.830294 22829 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.830734 22824 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.834637 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.835638 16594 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:24.802850 16587 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:24.839556 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.840252 17414 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:24.842717 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:24.843566 17414 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:24.845218 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.842967 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.845978 17420 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:20:24.849038 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 153655us
I20260709 13:20:24.837981 22823 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:24.848048 16478 catalog_manager.cc:5028] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): tablet 81853b1e217c4b3aa8033dd7e0fdf211 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:20:24.849676 22792 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:24.873031 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:24.875181 22792 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:24.876274 22792 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.841421 22822 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.881031 16487 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:24.891029 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.899276 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:24.903995 22848 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:24.904951 22793 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:24.905640 22793 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.900503 22681 raft_consensus.cc:2764] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:20:24.907881 22793 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.908610 22834 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:24.908995 22834 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:24.909348 22834 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.914219 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:24.915728 22793 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:24.915825 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.916199 22793 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:24.916661 22793 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:24.919760 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.879823 22842 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:24.923645 22842 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.926338 22842 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.928164 22822 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:24.865314 22823 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:24.929710 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:24.930672 16590 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.931677 22823 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:24.933563 16667 raft_consensus.cc:2368] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:20:24.935322 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:24.935540 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.936012 16664 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.937758 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:24.938408 16594 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:24.941259 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.942112 17414 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:24.951210 16664 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:24.882135 16592 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:24.953205 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:24.962636 16743 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:24.968657 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.968778 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:24.969763 17409 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:24.972051 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:24.972359 16592 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:24.969308 17420 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:24.976804 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.977283 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.980448 16667 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.985302 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:24.990754 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:24.989869 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:24.993391 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:24.994060 16742 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:20:24.993772 16667 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:24.996827 16739 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:25.001886 16559 leader_election.cc:400] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:20:25.002897 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:24.990923 22782 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:25.009143 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:25.019443 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:25.021535 22832 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:25.021621 22692 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:25.022169 22832 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:25.022349 22692 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:25.021171 22786 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.023613 16668 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:25.023109 22832 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.023375 22692 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.018361 17409 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:25.029565 22824 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:25.033231 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:25.035463 22822 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:25.035439 22820 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.036170 22822 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:25.037817 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:25.040064 22820 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:25.040562 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:25.038623 16739 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:25.044224 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:25.013209 22782 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:25.046904 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:25.047596 16736 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.061287 22685 raft_consensus.cc:993] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:25.061846 22685 raft_consensus.cc:1081] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:25.066638 22834 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.067937 16588 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:20:25.068924 22834 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:25.049759 22782 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:25.086575 22822 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.087942 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:20:25.119277 16593 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:25.119916 16593 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.107717 16742 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
W20260709 13:20:25.125699 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 133368us
I20260709 13:20:25.109230 16736 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:25.130048 16739 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
W20260709 13:20:25.144610 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 464776us
W20260709 13:20:25.148864 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 104724us
I20260709 13:20:25.156569 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:25.157449 16667 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:25.104610 22793 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.163439 16708 leader_election.cc:400] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:25.163965 16708 leader_election.cc:403] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:20:25.172278 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 169474us
I20260709 13:20:25.175617 16596 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:20:25.174391 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:25.191287 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:25.198555 17409 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Advancing to term 17
I20260709 13:20:25.206071 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:25.209275 17409 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:25.216488 16662 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:25.190394 16667 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:20:25.225879 22832 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:25.224718 22851 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:25.190374 22789 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.238023 16593 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.244546 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:25.245142 16662 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:25.245409 16662 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:25.251523 16664 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:25.253786 16663 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:25.254670 17414 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:25.239874 22832 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.257774 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:25.266455 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:25.268575 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:25.282142 16663 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:25.299062 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:25.299649 16590 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:25.299747 16591 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:25.267330 22832 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.301420 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:25.246479 22789 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.308194 16594 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:25.281916 22851 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:25.307741 16592 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:20:25.287601 22840 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:25.331355 16590 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:25.288228 22813 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:20:25.313841 16662 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:25.334684 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62511.2us
W20260709 13:20:25.336426 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 101970us
I20260709 13:20:25.324690 22851 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:25.325827 22840 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:25.343345 16591 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:20:25.363942 16662 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:25.337634 22681 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.339004 22793 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:25.335225 22849 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:20:25.384337 22782 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.388772 16487 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:25.394814 22789 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:25.369594 22866 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:20:25.376133 22840 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:25.438333 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 72871us
I20260709 13:20:25.440457 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:25.443356 22789 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:25.445182 22872 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:25.445660 22872 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:25.446470 22872 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.447243 16736 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:20:25.448895 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:25.449553 17414 raft_consensus.cc:2368] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:20:25.447170 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:25.451188 22707 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:25.451365 22841 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:25.386777 22782 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:25.457887 16592 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:25.460202 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:25.474005 17409 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:25.474514 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:25.461473 22782 raft_consensus.cc:2764] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:20:25.396044 22681 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.477545 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:25.478238 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:25.480630 16561 leader_election.cc:400] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
I20260709 13:20:25.480927 16561 leader_election.cc:403] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:25.489573 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:25.501798 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:25.516901 22856 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:25.519613 22862 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:25.520232 22862 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:25.524442 22871 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:20:25.525067 22832 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.525484 22879 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:25.534389 16662 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:25.511435 22786 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:25.526779 22788 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:25.527406 22681 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:25.510298 22841 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 13:20:25.578933 22876 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:20:25.395773 22851 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:25.580158 22865 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:25.580826 22865 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:25.581763 22865 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.521194 22862 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.570294 22788 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:25.583966 22788 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:25.591825 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:20:25.657487 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 155756us
W20260709 13:20:25.661651 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 172182us
I20260709 13:20:25.661711 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:25.670722 16741 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:20:25.671424 22880 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:20:25.675972 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:25.681638 22792 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:25.703212 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:25.706110 16736 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:26.153343 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4055) took 1093 ms. Trace:
W20260709 13:20:26.221326 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.220980 16739 rpcz_store.cc:276] 0709 13:20:25.060163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.100005 (+ 39842us) service_pool.cc:224] Handling call
0709 13:20:25.129762 (+ 29757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:25.130593 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.153076 (+1022483us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b54003d4850. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:26.153244 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66945280}
I20260709 13:20:26.152249 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 1176) took 1237 ms. Trace:
W20260709 13:20:26.051903 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 231996us
I20260709 13:20:26.222479 16588 rpcz_store.cc:276] 0709 13:20:24.915195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:24.915346 (+   151us) service_pool.cc:224] Handling call
0709 13:20:26.152009 (+1236663us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54000d9dd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:26.152135 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348045184}
I20260709 13:20:26.222489 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:26.248096 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.017s
W20260709 13:20:26.251791 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.018s
W20260709 13:20:26.252259 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 562140us
W20260709 13:20:26.219610 22856 consensus_meta.cc:287] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.004s
W20260709 13:20:26.254264 16590 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.006s	sys 0.009s
W20260709 13:20:26.254624 16590 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.006s	sys 0.009s
I20260709 13:20:26.084398 16736 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:25.950709 22832 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:26.120711 22786 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:26.120111 22874 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:26.163697 22884 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:20:25.705113 22866 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.566s
I20260709 13:20:26.190116 22685 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.263614 22856 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13.
W20260709 13:20:26.297389 16636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.037s	sys 0.014s
W20260709 13:20:26.297766 16636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.037s	sys 0.015s
I20260709 13:20:25.820322 22792 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:25.713099 22791 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:26.107842 22895 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:26.085412 22862 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.308689 16588 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.097419 22788 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.322820 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:26.330284 17416 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:26.332248 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:26.333158 16665 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:26.333197 22825 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:26.333314 22890 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:26.353286 17410 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:26.295595 22786 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.307936 22824 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:26.337181 22894 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:20:26.299727 22685 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.333614 22792 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:26.298650 22874 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.378675 22881 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.392262 22868 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:26.392416 22881 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.392896 22868 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:26.395928 16593 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.396440 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38576 (request call id 4455) took 1331 ms. Trace:
I20260709 13:20:26.396768 16593 rpcz_store.cc:276] 0709 13:20:25.064666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.087827 (+ 23161us) service_pool.cc:224] Handling call
0709 13:20:26.395307 (+1307480us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b4c00367d40. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327deb0a20 00007f327dec1a29 00007f327dec213f 00007f327de862ab 00007f327deb3f3c 00007f327deb43fe 00007f327e14188b 00007f327fb40495 00007f327fb9130f 00007f327fbdba55 00007f3284babcd0
0709 13:20:26.396363 (+  1056us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20336512}
I20260709 13:20:26.370801 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:26.323174 22862 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.399551 22862 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.399633 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:26.400352 16666 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:26.400689 16666 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:26.357208 22825 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.401371 16666 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.402537 16666 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:26.402851 22825 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:26.403254 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:26.311918 22791 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.403913 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:26.404399 16665 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:26.404513 16669 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:26.405377 22791 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:26.370826 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:26.425806 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:26.427770 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:26.429977 17420 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:26.433041 16667 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:26.334956 22821 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:26.426991 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56294.8us
I20260709 13:20:26.440418 16668 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:26.394698 22881 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.441190 17416 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:26.445072 22821 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.320858 22895 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:26.447093 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:26.448700 22901 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.455842 22788 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.456058 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:26.515827 17420 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:26.462045 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:26.464216 16667 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:26.467357 16666 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
W20260709 13:20:26.518121 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56183.4us
I20260709 13:20:26.475899 16593 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:26.482419 22695 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:26.482728 22874 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.485271 22832 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:26.489499 22909 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:26.491830 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:26.495849 22786 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:26.497319 22868 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.370246 22824 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.499181 22895 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:26.501967 22821 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:26.530210 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 63993.7us
I20260709 13:20:26.441087 22866 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.443341 22758 raft_consensus.cc:2764] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:20:26.699296 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 411730us
W20260709 13:20:26.568745 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48666.3us
I20260709 13:20:26.697232 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:26.706439 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:26.747190 16590 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:26.751132 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:26.534976 22849 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:26.777431 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:26.779189 16666 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:20:26.846994 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 140671us
I20260709 13:20:26.849587 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:26.759500 17416 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:26.806365 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:26.851568 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:26.852898 17420 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:26.706190 22890 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:26.778348 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3006) took 1587 ms. Trace:
I20260709 13:20:26.869460 16592 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:20:26.703425 22881 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:26.872041 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 80059.9us
I20260709 13:20:26.858934 16671 rpcz_store.cc:276] 0709 13:20:25.190495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.225253 (+ 34758us) service_pool.cc:224] Handling call
0709 13:20:26.750360 (+1525107us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b5800210998. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:20:26.777647 (+ 27287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:26.777881 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.777912 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:26.777964 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:26.778004 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:26.778059 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:26.778183 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14334848}
I20260709 13:20:26.870710 16587 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:26.887701 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:26.895781 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:26.803730 22876 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:20:26.898211 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:26.841741 22889 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.761155 22839 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.856735 22862 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:26.750348 22788 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:26.750527 22753 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.806653 22911 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.778679 22834 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.732910 22832 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.842300 22840 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.841441 22888 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.802388 22847 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.807603 22707 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.778831 22856 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:26.870702 22890 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:26.937268 16665 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.006s	sys 0.015s
I20260709 13:20:26.537210 22901 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.802039 22906 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.208s
I20260709 13:20:26.802708 22900 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.202s
I20260709 13:20:26.777750 22866 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.931977 22879 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:20:26.890025 22849 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:27.176970 16665 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.011s	sys 0.027s
I20260709 13:20:26.931432 22889 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.148651 22764 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.935353 22839 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:27.164641 22921 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:20:26.985790 22890 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:27.162668 22864 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.168061 22681 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:27.162024 22813 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.171298 22843 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.204249 22843 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.206926 22813 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.209141 22813 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.209941 22813 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.210431 22813 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.213212 16487 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 311) took 1983 ms. Trace:
I20260709 13:20:27.213497 22681 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.215612 22681 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.218210 22889 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.219290 16669 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:20:27.152210 22786 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.192065 16741 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:20:27.201928 22917 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.202425 22932 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.202986 22924 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.203712 22929 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:26.998395 17413 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:27.237618 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:27.233531 22758 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.187691 22856 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.993736 22707 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.252839 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:26.968952 22851 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.940562 22862 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:26.961546 22848 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:27.299216 22848 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.999264 22846 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.290194 22832 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.322566 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:26.993489 22847 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.240971 22908 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:27.347934 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.186849 22764 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.941009 22788 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.945758 22881 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.350428 22923 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.353539 22844 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.390012 22844 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.391525 22866 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.392598 22813 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.392958 22844 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:26.999032 22918 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.300465 22824 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.383369 22839 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.323444 22695 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:20:27.242307 22917 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.247714 22924 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.387135 22908 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:26.948176 22909 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.213656 16487 rpcz_store.cc:276] 0709 13:20:25.229502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:25.229722 (+   220us) service_pool.cc:224] Handling call
0709 13:20:27.213118 (+1983396us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:25.469433 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:25.469981 (+   548us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:25.470012 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:20:25.470029 (+    17us) tablet.cc:662] Decoding operations
0709 13:20:25.470333 (+   304us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:25.470400 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:25.470428 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:25.470595 (+   167us) tablet.cc:701] Row locks acquired
0709 13:20:25.470610 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:20:25.470741 (+   131us) write_op.cc:270] Start()
0709 13:20:25.470900 (+   159us) write_op.cc:276] Timestamp: P: 1783603225470696 usec, L: 0
0709 13:20:25.470924 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:20:25.540295 (+ 69371us) log.cc:844] Serialized 530 byte log entry
0709 13:20:25.655135 (+114840us) op_driver.cc:464] REPLICATION: finished
0709 13:20:26.337324 (+682189us) write_op.cc:301] APPLY: starting
0709 13:20:26.337413 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:26.338586 (+  1173us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:26.338673 (+    87us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:26.339374 (+   701us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:26.339549 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:26.339591 (+    42us) write_op.cc:312] APPLY: finished
0709 13:20:26.339776 (+   185us) log.cc:844] Serialized 25 byte log entry
0709 13:20:26.339927 (+   151us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:26.340001 (+    74us) write_op.cc:454] Released schema lock
0709 13:20:26.340040 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:26.340424 (+   384us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:26.340448 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":226176,"child_traces":[["op",{"apply.queue_time_us":655923,"apply.run_cpu_time_us":3121,"apply.run_wall_time_us":15894,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"mutex_wait_us":28603,"num_ops":1,"prepare.queue_time_us":58628,"prepare.run_cpu_time_us":2953,"prepare.run_wall_time_us":97871,"replication_time_us":184040,"thread_start_us":195127,"threads_started":2}]]}
I20260709 13:20:26.989624 22888 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.240919 22758 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.266343 22681 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:26.979230 22840 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.444299 22789 raft_consensus.cc:1064] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:20:27.444819 22789 raft_consensus.cc:1068] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Unable to promote non-voter ba2df6a668144022a223e504ae571f21: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:20:27.445924 22840 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.446113 22903 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.446949 16744 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:26.964699 22911 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:27.414923 16664 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.040s	sys 0.083s
W20260709 13:20:27.455485 16664 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.041s	sys 0.083s
I20260709 13:20:27.354650 22933 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:26.973408 22834 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.242933 22786 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:26.947727 22753 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.427718 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:27.200635 22864 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.399698 22906 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.259128 22932 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.442690 22707 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.443615 22681 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.404906 22537 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:27.418992 22813 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.346743 22851 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:27.460225 22933 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.461019 22911 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.461220 22914 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:27.461750 22779 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:27.462162 22928 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:20:27.462685 22922 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.461064 22902 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.520664 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3019) took 1119 ms. Trace:
I20260709 13:20:27.472003 22924 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.475978 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1493) took 1093 ms. Trace:
I20260709 13:20:27.476243 22908 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.480561 22758 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.481249 22848 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.483130 22881 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.478510 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1263) took 1155 ms. Trace:
I20260709 13:20:27.289956 22929 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.446705 22903 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.497920 22537 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:27.408298 22923 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.490295 22847 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.348096 22846 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.439980 22909 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.411734 22918 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.436683 22862 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.525985 16664 rpcz_store.cc:276] 0709 13:20:26.401299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.405218 (+  3919us) service_pool.cc:224] Handling call
0709 13:20:26.405536 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.434848 (+ 29312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.434929 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:26.444822 (+  9893us) log.cc:844] Serialized 26 byte log entry
0709 13:20:27.455973 (+1011151us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:27.456032 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.456155 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:27.520446 (+ 64291us) raft_consensus.cc:1685] finished
0709 13:20:27.520467 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.520604 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19325,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":14626,"thread_start_us":945069,"threads_started":1,"wal-append.queue_time_us":981280}
W20260709 13:20:27.586375 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:27.549808 16743 rpcz_store.cc:276] 0709 13:20:26.382117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.386799 (+  4682us) service_pool.cc:224] Handling call
0709 13:20:26.387066 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:26.412512 (+ 25446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.412554 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:26.412612 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:26.412651 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.475658 (+1063007us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b54002b93d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:27.475734 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.475893 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58006272}
I20260709 13:20:27.493386 22782 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.434942 22890 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.517261 22888 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.617800 22943 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.620249 22537 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:27.620672 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.002s	sys 0.008s
I20260709 13:20:27.616225 17409 rpcz_store.cc:276] 0709 13:20:26.322784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.322933 (+   149us) service_pool.cc:224] Handling call
0709 13:20:26.323208 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:26.809132 (+485924us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.809176 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:26.809222 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:26.809263 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.478168 (+668905us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400549cd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:27.478219 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.478398 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1117006592}
I20260709 13:20:27.516899 22786 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.511365 22813 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.588300 22914 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.592306 22902 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.628531 22933 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.516844 22681 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.520989 22753 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.508961 22707 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.713591 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:27.713948 22827 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:27.713811 22707 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.714327 17412 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:27.714394 22827 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:27.518610 22834 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.514400 22917 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.515606 22788 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.629189 16745 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:20:27.715317 22914 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.715824 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 1496) took 1021 ms. Trace:
I20260709 13:20:27.715438 22827 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.716169 16741 rpcz_store.cc:276] 0709 13:20:26.694889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.697025 (+  2136us) service_pool.cc:224] Handling call
0709 13:20:27.715625 (+1018600us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400427054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:27.715760 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":785983872}
I20260709 13:20:27.716606 22827 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:27.717317 22813 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.717420 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:27.718016 16741 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:27.718602 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:27.719812 22902 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.719918 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:27.523510 22901 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.645965 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:27.648397 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.003s	sys 0.008s
I20260709 13:20:27.724474 16743 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:27.649225 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:27.651126 22537 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.657071 22909 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.639853 22881 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:27.517947 22779 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.647351 22782 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.664141 22764 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.588554 22922 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.676880 22918 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.678686 22932 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.680591 16592 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:27.682214 22890 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.683773 22851 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.685515 22923 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:27.687841 22864 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.498018 22906 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.692523 22929 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.699822 22943 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.726163 22537 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:27.724344 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:27.753652 22881 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:27.805924 22843 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.731599 22853 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.727321 22825 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.762318 22788 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.767866 22782 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.759013 22786 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:27.781438 22922 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:27.781821 22849 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:27.746658 22901 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:20:27.726913 22707 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:27.820047 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 227115us
I20260709 13:20:27.824039 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:27.826499 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:27.807790 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:27.831569 22881 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.107666 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.221985 17413 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:20:28.108529 22537 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:28.187686 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.222838 22537 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:27.947110 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.213683 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.065157 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:28.224764 17415 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
W20260709 13:20:28.110486 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.221843 16588 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
W20260709 13:20:28.229393 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 395665us
I20260709 13:20:28.230278 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:28.098706 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3484) took 1285 ms. Trace:
I20260709 13:20:27.969316 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4461) took 1456 ms. Trace:
I20260709 13:20:27.964529 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.237763 17412 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:27.841264 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.096217 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 3206) took 1246 ms. Trace:
I20260709 13:20:28.066347 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4468) took 1085 ms. Trace:
I20260709 13:20:28.242014 22537 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.242224 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:27.801330 22906 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.243373 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:28.245625 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.246685 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.054302 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 947) took 1302 ms. Trace:
I20260709 13:20:27.965827 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4465) took 1200 ms. Trace:
I20260709 13:20:28.239668 16589 rpcz_store.cc:276] 0709 13:20:26.849502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.936901 (+ 87399us) service_pool.cc:224] Handling call
0709 13:20:27.148298 (+211397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.347531 (+199233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.347570 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.347608 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:27.347811 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.095789 (+747978us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b540010d954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.095855 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.096017 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457705216}
I20260709 13:20:28.240386 16594 rpcz_store.cc:276] 0709 13:20:26.980974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.168724 (+187750us) service_pool.cc:224] Handling call
0709 13:20:27.474330 (+305606us) spinlock_profiling.cc:243] Waited 9.28 ms on lock 0x7b5800010e98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:20:27.474566 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.783461 (+308895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.783506 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.783549 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:27.783581 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.066036 (+282455us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b540033a9d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.066102 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.066272 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731238528}
I20260709 13:20:28.255623 16596 rpcz_store.cc:276] 0709 13:20:26.765339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.855802 (+ 90463us) service_pool.cc:224] Handling call
0709 13:20:26.869733 (+ 13931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.354535 (+484802us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.354562 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.354604 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:27.354634 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.965499 (+610865us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54001faed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:27.965559 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.965720 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1114508032}
W20260709 13:20:28.248884 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.248764 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.260711 16670 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:28.261806 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.260763 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.231849 16669 rpcz_store.cc:276] 0709 13:20:26.813533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.813665 (+   132us) service_pool.cc:224] Handling call
0709 13:20:26.846719 (+ 33054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.219752 (+373033us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.098424 (+878672us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54001b08d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.098629 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":855851264}
I20260709 13:20:28.236047 16591 rpcz_store.cc:276] 0709 13:20:26.513395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.513525 (+   130us) service_pool.cc:224] Handling call
0709 13:20:26.529853 (+ 16328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.194390 (+664537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.194419 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.194478 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:20:27.194519 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.969002 (+774483us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b5400245254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:27.969057 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:27.969238 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1527878528}
I20260709 13:20:28.244599 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.250924 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 4470) took 1038 ms. Trace:
I20260709 13:20:28.247641 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.265664 16588 rpcz_store.cc:276] 0709 13:20:27.190576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.211227 (+ 20651us) service_pool.cc:224] Handling call
0709 13:20:28.107484 (+896257us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b5800010e98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284baa0a0 00007f3284ba9984 00007f327e1246d4 00007f327e124662
0709 13:20:28.229256 (+121772us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20291840}
W20260709 13:20:28.228966 16634 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.621s (SENT) [suppressed 5 similar messages]
I20260709 13:20:28.267580 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:27.819550 22917 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.269320 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:28.061717 22913 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 1.302s	user 0.004s	sys 0.000s
I20260709 13:20:28.269961 17409 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:20:27.922014 22868 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:28.275403 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.270259 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.271709 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.272733 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:28.276201 16666 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:28.273339 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.253803 17411 rpcz_store.cc:276] 0709 13:20:26.752059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.752180 (+   121us) service_pool.cc:224] Handling call
0709 13:20:26.752491 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:26.856047 (+103556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:26.856078 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:27.628986 (+772908us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b2c0004cd68. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:20:27.647135 (+ 18149us) log.cc:844] Serialized 26 byte log entry
0709 13:20:27.728575 (+ 81440us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:27.950568 (+221993us) log.cc:844] Serialized 14 byte log entry
0709 13:20:27.951078 (+   510us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.951201 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:27.951267 (+    66us) raft_consensus.cc:1685] finished
0709 13:20:28.054009 (+102742us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.054211 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27779,"prepare.queue_time_us":485857,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":76995,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":69698,"spinlock_wait_cycles":33486720,"thread_start_us":272891,"threads_started":2,"wal-append.queue_time_us":175733}
I20260709 13:20:27.970743 22685 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.283684 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:27.935760 22864 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:28.279125 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.280711 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:27.945608 22692 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:27.839265 22856 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.299897 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.300132 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.206691 22909 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.308845 22903 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.309120 22909 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:28.310171 16667 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:28.240140 22922 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.311193 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.311467 22922 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.312368 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.312868 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.313453 22922 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:28.314237 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.316032 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.317152 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.317364 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.318006 16667 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:28.318082 16590 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:20:27.967837 22792 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:28.319811 17420 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:28.320950 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:28.317178 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 3207) took 1446 ms. Trace:
I20260709 13:20:28.324151 22537 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:28.324752 22537 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:28.288956 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58835.8us
I20260709 13:20:28.281916 22917 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.291769 22864 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:28.293219 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.284528 22857 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:28.285636 16742 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:28.250363 22865 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:28.277307 22937 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.336037 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.336923 16588 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:27.847064 22788 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.298900 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.301355 22758 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:28.301896 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.278021 22858 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:20:28.380489 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.295384 22707 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:27.905498 22825 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:28.339887 22901 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:28.380035 22871 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:20:28.307848 22943 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.395480 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4066) took 1028 ms. Trace:
I20260709 13:20:28.399879 22864 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.399246 16739 rpcz_store.cc:276] 0709 13:20:27.367415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.367531 (+   116us) service_pool.cc:224] Handling call
0709 13:20:27.367832 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.448288 (+ 80456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.448318 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.448356 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:27.448384 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.395258 (+946874us) spinlock_profiling.cc:243] Waited 80.3 ms on lock 0x7b54003d4354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.395306 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.395421 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184603392}
I20260709 13:20:28.400604 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.400960 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.308507 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.338281 22938 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.401999 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.402444 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.403081 16664 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:28.403162 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.403818 16744 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:28.404106 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.404839 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.313354 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.279789 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 4472) took 1014 ms. Trace:
I20260709 13:20:28.405867 16594 rpcz_store.cc:276] 0709 13:20:27.265732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.273120 (+1007388us) service_pool.cc:224] Handling call
0709 13:20:28.279730 (+  6610us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:28.326452 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.407477 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.314527 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.319121 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:28.283918 22948 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:20:28.322197 16592 rpcz_store.cc:276] 0709 13:20:26.870857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.871893 (+  1036us) service_pool.cc:224] Handling call
0709 13:20:26.872231 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.754143 (+881912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.316956 (+562813us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b54002c7c54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.317121 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858659072}
I20260709 13:20:27.847445 22786 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.395982 22707 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.439508 22707 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.343784 22937 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.442427 22937 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.420043 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.272187 22900 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:28.297233 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.282316 22782 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:28.331357 22862 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.332059 22857 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:28.340894 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:28.341820 16668 raft_consensus.cc:2393] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:27.847577 22853 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.295651 22793 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.277674 16596 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:28.343473 16666 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:28.367524 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.382004 16663 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:28.335297 22685 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:28.280460 22868 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:28.383643 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.328253 22827 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.332993 22692 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:28.338011 22899 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.402043 22938 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.413965 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.418673 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:28.395232 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.405884 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.063306 22779 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:28.402083 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.412453 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.451052 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:28.762378 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 421544us
I20260709 13:20:28.718039 17414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1748) took 1068 ms. Trace:
I20260709 13:20:28.681568 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4469) took 1503 ms. Trace:
I20260709 13:20:28.535209 22537 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:28.675872 16741 consensus_meta.cc:287] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.957s	user 0.016s	sys 0.002s
I20260709 13:20:28.768244 16593 rpcz_store.cc:276] 0709 13:20:27.178524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.677195 (+1498671us) service_pool.cc:224] Handling call
0709 13:20:28.680920 (+  3725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.681146 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.681183 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.681238 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:28.681281 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.681345 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.681488 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.763803 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 4473) took 1447 ms. Trace:
I20260709 13:20:28.770164 16596 rpcz_store.cc:276] 0709 13:20:27.316609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.260165 (+943556us) service_pool.cc:224] Handling call
0709 13:20:28.763746 (+503581us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.761377 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.756268 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.574385 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.772070 17416 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
W20260709 13:20:28.773053 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 500363us
I20260709 13:20:28.461061 16587 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:28.567219 16670 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:28.448108 16736 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:28.569953 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.766986 16665 raft_consensus.cc:606] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:28.769418 16741 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
W20260709 13:20:28.776731 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 547769us
I20260709 13:20:28.777248 16744 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:20:28.760921 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.713091 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4068) took 1313 ms. Trace:
I20260709 13:20:28.759912 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.827726 22537 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:28.568576 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4073) took 1065 ms. Trace:
I20260709 13:20:28.850520 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 2543) took 1069 ms. Trace:
I20260709 13:20:28.767428 17414 rpcz_store.cc:276] 0709 13:20:27.649661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.649780 (+   119us) service_pool.cc:224] Handling call
0709 13:20:27.655271 (+  5491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.761833 (+106562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.761879 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.761943 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:27.761989 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.717784 (+955795us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54000e33d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.717833 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.717981 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244499200}
I20260709 13:20:28.869680 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.873178 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.890537 16666 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:20:28.891966 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.867518 16741 rpcz_store.cc:276] 0709 13:20:27.707846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.717156 (+  9310us) service_pool.cc:224] Handling call
0709 13:20:28.777451 (+1060295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.342957 22758 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:28.794718 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:28.906232 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.872969 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:28.765956 22857 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:28.779445 22862 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.911592 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.912859 22862 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.913035 16738 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:28.914160 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.915516 22926 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.916097 22926 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.913045 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:28.571192 22909 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:28.450620 22950 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:20:28.898033 16664 raft_consensus.cc:2410] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:20:28.782821 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.898403 22537 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:28.893710 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1205) took 1401 ms. Trace:
I20260709 13:20:28.563261 22943 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:28.867393 22856 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 13:20:28.831858 22947 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:28.932144 22943 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.932337 22947 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.898973 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:28.935158 17409 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:20:28.936960 22857 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.938055 22692 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.831544 22793 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.939816 22857 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.941112 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2545) took 1232 ms. Trace:
I20260709 13:20:28.941660 22853 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.942263 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.581208 22851 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.903975 16595 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:28.395787 22846 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.679649 22901 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.779069 22906 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:20:28.711373 22936 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.095s	user 0.005s	sys 0.000s
I20260709 13:20:28.778625 16737 rpcz_store.cc:276] 0709 13:20:27.407205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.455376 (+ 48171us) service_pool.cc:224] Handling call
0709 13:20:28.712115 (+1256739us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b58001d5b98. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cabfa22 00007f32859d8c09 00007f3284b32ee0 00007f3284bfc0ab 00007f3284bfc131 00007f3284bbdfc0 00007f3284ba8929 00007f3284ba83d2 00007f327e11fbd4
0709 13:20:28.712352 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.712560 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.712590 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.712635 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:28.712740 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.712802 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.712920 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11346304}
I20260709 13:20:28.919656 16590 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:28.319299 22792 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.779335 22929 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.723088 22786 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.340852 22865 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.573729 22917 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.925582 16594 rpcz_store.cc:276] 0709 13:20:27.491856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.892980 (+1401124us) service_pool.cc:224] Handling call
0709 13:20:28.893255 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.893424 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.893448 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.893486 (+    38us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:28.893519 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.893567 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.893658 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.779201 22707 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:28.930447 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:28.865633 22685 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:28.941506 16743 rpcz_store.cc:276] 0709 13:20:27.708624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.063351 (+354727us) service_pool.cc:224] Handling call
0709 13:20:28.063854 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.280996 (+217142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.281026 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.283413 (+  2387us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.717986 (+434573us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:28.718026 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.765084 (+ 47058us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.940932 (+175848us) raft_consensus.cc:1685] finished
0709 13:20:28.940948 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.941057 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216684,"raft.queue_time_us":239171,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"thread_start_us":434170,"threads_started":1,"wal-append.queue_time_us":655775}
I20260709 13:20:28.867357 16740 rpcz_store.cc:276] 0709 13:20:27.503210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.515168 (+ 11958us) service_pool.cc:224] Handling call
0709 13:20:27.515417 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:27.675504 (+160087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.675550 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:27.675612 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:27.675670 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.568067 (+892397us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54003edfd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.568157 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.568356 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367365888}
I20260709 13:20:28.446478 22825 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:28.718158 22849 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:28.917398 16668 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 9.
I20260709 13:20:28.944216 17419 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:28.948567 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4471) took 1728 ms. Trace:
I20260709 13:20:28.975842 16595 rpcz_store.cc:276] 0709 13:20:27.220014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.779674 (+1559660us) service_pool.cc:224] Handling call
0709 13:20:28.899446 (+119772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.946018 (+ 46572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.948510 (+  2492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.977533 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1210) took 1358 ms. Trace:
I20260709 13:20:28.977885 16595 rpcz_store.cc:276] 0709 13:20:27.619451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.976649 (+1357198us) service_pool.cc:224] Handling call
0709 13:20:28.976994 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.977194 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.977223 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.977266 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:28.977309 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.977363 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.977480 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.864820 22791 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:28.979153 22791 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.981428 22791 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.984440 22792 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.986150 22943 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:28.991911 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4475) took 1597 ms. Trace:
I20260709 13:20:28.992264 22938 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.992323 16590 rpcz_store.cc:276] 0709 13:20:27.394050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.832991 (+1438941us) service_pool.cc:224] Handling call
0709 13:20:28.907506 (+ 74515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.952382 (+ 44876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.991702 (+ 39320us) spinlock_profiling.cc:243] Waited 9.84 ms on lock 0x7b54003584d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:28.991854 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22625152}
I20260709 13:20:28.993999 22926 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.994114 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 3210) took 1177 ms. Trace:
I20260709 13:20:28.995601 22868 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.996744 22788 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.955996 22906 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:28.998198 22906 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.002633 22849 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.004380 16487 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 16, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 16 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:29.006131 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2547) took 1051 ms. Trace:
I20260709 13:20:29.006539 16745 rpcz_store.cc:276] 0709 13:20:27.955197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:27.955551 (+   354us) service_pool.cc:224] Handling call
0709 13:20:28.068014 (+112463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.298957 (+230943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.298985 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.452531 (+153546us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b2c00141498. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:20:28.453081 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:20:28.951350 (+498269us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:28.951386 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.951518 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.005826 (+ 54308us) raft_consensus.cc:1685] finished
0709 13:20:29.005845 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.006071 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":230120,"raft.queue_time_us":249437,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":73061760,"thread_start_us":497859,"threads_started":1,"wal-append.queue_time_us":550969}
I20260709 13:20:29.008478 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.010979 22707 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.011795 22865 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:28.966939 22929 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.013283 22707 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.014510 22929 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:28.917328 22758 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:28.831184 22899 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.952931 22537 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:28.951594 16636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.030s
I20260709 13:20:28.923547 22843 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 13:20:29.031455 16636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.005s	sys 0.030s
W20260709 13:20:29.031908 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 684466us
I20260709 13:20:28.954433 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:28.780619 22827 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.034686 16742 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:20:28.869267 22782 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.958544 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1209) took 1454 ms. Trace:
I20260709 13:20:29.049635 16594 rpcz_store.cc:276] 0709 13:20:27.504643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.957435 (+1452792us) service_pool.cc:224] Handling call
0709 13:20:28.957867 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.958105 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.958143 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.958202 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:28.958258 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.958337 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.958486 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.994469 16590 rpcz_store.cc:276] 0709 13:20:27.816799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.993343 (+1176544us) service_pool.cc:224] Handling call
0709 13:20:28.993656 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.993816 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.993844 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.993890 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:28.993919 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.993970 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.994062 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.959386 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.014688 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.015543 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.052872 16743 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:28.951592 22851 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.015945 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.009629 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.017391 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3031) took 2054 ms. Trace:
I20260709 13:20:28.950143 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1207) took 1453 ms. Trace:
I20260709 13:20:29.063974 16593 rpcz_store.cc:276] 0709 13:20:27.496335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.920600 (+1424265us) service_pool.cc:224] Handling call
0709 13:20:28.947629 (+ 27029us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.947855 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.947893 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:28.947948 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:28.947996 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:28.949941 (+  1945us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:28.950076 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.017567 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.065214 16589 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:29.065814 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 3215) took 1153 ms. Trace:
I20260709 13:20:29.066179 16589 rpcz_store.cc:276] 0709 13:20:27.912986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.017314 (+1104328us) service_pool.cc:224] Handling call
0709 13:20:29.065739 (+ 48425us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:28.960065 22901 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:28.955506 22692 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.020150 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.080860 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.081544 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.082177 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.106333 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.107782 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.108943 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:29.082641 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 404304us
I20260709 13:20:29.084334 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.115934 16662 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:20:29.085114 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.081123 22537 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:29.117843 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:29.060915 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:29.052471 22869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.012s	sys 0.011s
I20260709 13:20:29.122751 17416 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:29.121860 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 2555) took 1046 ms. Trace:
I20260709 13:20:29.123407 16737 rpcz_store.cc:276] 0709 13:20:28.066677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.953187 (+886510us) service_pool.cc:224] Handling call
0709 13:20:29.112867 (+159680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:29.119279 16592 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:20:29.120112 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.126210 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.127019 16665 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:29.062759 16671 rpcz_store.cc:276] 0709 13:20:26.963372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.972829 (+  9457us) service_pool.cc:224] Handling call
0709 13:20:27.001784 (+ 28955us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:27.145480 (+143696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:27.145545 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:27.179040 (+ 33495us) log.cc:844] Serialized 26 byte log entry
0709 13:20:27.360090 (+181050us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:27.360159 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:27.360252 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:28.706990 (+1346738us) spinlock_profiling.cc:243] Waited 5.43 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:29.017108 (+310118us) raft_consensus.cc:1685] finished
0709 13:20:29.017131 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.017328 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140942,"raft.queue_time_us":142044,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"spinlock_wait_cycles":12484864,"thread_start_us":132036,"threads_started":1,"wal-append.queue_time_us":285268}
I20260709 13:20:29.129123 16668 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:29.129360 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.129963 16666 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:20:29.049712 22929 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:29.141254 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:29.122947 22869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.013s	sys 0.011s
W20260709 13:20:29.139081 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 422006us
I20260709 13:20:29.164220 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.182602 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:29.197976 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 168410us
I20260709 13:20:29.231994 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38576 (request call id 4476) took 1592 ms. Trace:
I20260709 13:20:29.232540 16592 rpcz_store.cc:276] 0709 13:20:27.640318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.419825 (+779507us) service_pool.cc:224] Handling call
0709 13:20:29.231779 (+811954us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400244d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:29.231931 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554155904}
I20260709 13:20:29.201553 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.220976 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.244824 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.118652 22692 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.248407 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.251200 22846 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.251534 22895 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:29.251950 22846 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.253361 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.254490 22930 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.255024 22930 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.255476 22930 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.259191 22895 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.259748 22895 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.260166 22895 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.263401 16710 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.264496 22926 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.265000 22926 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.265440 22926 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:29.271606 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.279400 22764 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.279888 22764 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.280138 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1204) took 1975 ms. Trace:
I20260709 13:20:29.280370 22764 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.280751 16588 rpcz_store.cc:276] 0709 13:20:27.304604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.782036 (+1477432us) service_pool.cc:224] Handling call
0709 13:20:28.839230 (+ 57194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.102907 (+263677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.102942 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.102986 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:29.103026 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.279869 (+176843us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:29.279926 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.280073 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605882240}
I20260709 13:20:29.288329 22793 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.189632 22899 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.188383 22851 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.196223 22827 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.199129 22788 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.223596 22695 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.299813 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.170298 22865 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.229970 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.247689 22933 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.144335 22929 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.316699 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.316632 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.320199 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.324721 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.314353 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:29.320255 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 90356.5us
I20260709 13:20:29.328490 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.329064 17420 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:29.330914 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.137184 22685 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.334959 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.338403 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.341444 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:29.343660 16559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.020s	sys 0.034s
I20260709 13:20:29.344848 22537 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:29.338466 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:29.345124 16559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.020s	sys 0.034s
I20260709 13:20:29.347419 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:29.346956 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 513545us
I20260709 13:20:29.321794 22791 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.348109 17410 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:29.358469 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:29.355840 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 504615us
I20260709 13:20:29.321965 22851 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.360527 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.360989 16743 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:29.360666 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.372116 17420 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:20:29.374106 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.377857 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.378533 16665 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:20:29.333575 22957 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:29.374603 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.380846 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.379601 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:29.382347 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:29.384188 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 469035us
I20260709 13:20:29.386893 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.390506 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.391340 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.310324 22788 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.392216 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.312450 22695 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.389879 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.383018 16737 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:29.398027 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.347864 22846 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.320539 22865 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.371707 22909 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.356914 22791 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.331616 22933 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.372400 22851 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.355985 22827 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.408514 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.393404 22901 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.371784 22899 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.371842 22793 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:29.402063 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:29.383800 22957 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.392944 22833 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:29.405164 22707 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.426144 22707 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.426538 22707 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:29.407173 22791 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.408968 22851 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:29.401923 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.434273 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:29.435285 22964 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:20:29.425869 22833 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:29.441613 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.467288 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.468122 16741 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
W20260709 13:20:29.435783 16587 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 1206) took 1943 ms (client timeout 2160 ms). Trace:
I20260709 13:20:29.470842 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:29.472184 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 91430.6us
I20260709 13:20:29.434615 22764 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.474995 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.477029 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:29.476305 22764 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.481843 16741 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:29.479619 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.480965 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:29.477360 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.481863 16740 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:29.471280 16587 rpcz_store.cc:269] 0709 13:20:27.492563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.271432 (+778869us) service_pool.cc:224] Handling call
0709 13:20:29.435605 (+1164173us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540008acd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:29.435718 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423060352}
I20260709 13:20:29.429510 22917 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:29.407356 22909 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.493038 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3500) took 1184 ms. Trace:
I20260709 13:20:29.426149 22782 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:29.418655 22827 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:29.494761 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53242.4us
I20260709 13:20:29.507443 16667 rpcz_store.cc:276] 0709 13:20:28.308983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.420162 (+111179us) service_pool.cc:224] Handling call
0709 13:20:28.580536 (+160374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:28.580841 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:29.102661 (+521820us) log.cc:844] Serialized 14 byte log entry
0709 13:20:29.492690 (+390029us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.492751 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:29.492800 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.492874 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.492983 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62769,"prepare.queue_time_us":755322,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":63632,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":4518,"thread_start_us":736780,"threads_started":2}
I20260709 13:20:29.511682 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.425716 22957 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:29.455088 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:29.517060 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 135916us
I20260709 13:20:29.496778 16740 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:29.499166 22983 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.500893 22895 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.470580 22969 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:29.465929 22833 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:29.503135 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1278) took 1271 ms. Trace:
I20260709 13:20:29.494997 22868 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:29.572271 22910 raft_consensus.cc:1064] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:29.521585 22969 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.465830 22777 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:29.426091 22872 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:29.494659 16634 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:20:29.587244 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92660.8us
I20260709 13:20:29.592921 16670 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:20:29.425846 22899 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.595738 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.596321 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.596803 22899 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.596885 17416 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:29.597087 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.597682 17409 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:29.597874 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.504302 22695 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.529663 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.530383 16668 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:29.531253 22972 raft_consensus.cc:993] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:29.513916 22926 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.600930 22926 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.533591 22849 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:29.536547 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 958) took 1132 ms. Trace:
I20260709 13:20:29.539899 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 3209) took 1723 ms. Trace:
W20260709 13:20:29.606184 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 204192us
I20260709 13:20:29.606375 16595 rpcz_store.cc:276] 0709 13:20:27.816564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.978807 (+1162243us) service_pool.cc:224] Handling call
0709 13:20:29.435890 (+457083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:29.436067 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.436091 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:29.436133 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:29.436168 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.436225 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.539716 (+103491us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540008acd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:29.539837 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956838272}
I20260709 13:20:29.596804 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.607862 17414 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:29.606006 17412 rpcz_store.cc:276] 0709 13:20:28.403706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.934516 (+530810us) service_pool.cc:224] Handling call
0709 13:20:28.935095 (+   579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.935478 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.935507 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.936526 (+  1019us) log.cc:844] Serialized 487 byte log entry
0709 13:20:29.293597 (+357071us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:29.293644 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.293710 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.536285 (+242575us) raft_consensus.cc:1685] finished
0709 13:20:29.536312 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.536474 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":356748,"threads_started":1,"wal-append.queue_time_us":488767}
I20260709 13:20:29.540385 22827 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.541154 22929 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:29.541363 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.435755 22793 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:29.610500 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:29.612227 22793 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.548910 22788 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.549368 22692 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.550911 22895 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.551047 22909 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.507350 22917 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:29.613732 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.613953 22917 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:29.615085 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.527745 17413 rpcz_store.cc:276] 0709 13:20:28.231494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.260084 (+ 28590us) service_pool.cc:224] Handling call
0709 13:20:28.281223 (+ 21139us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:28.777302 (+496079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:28.777332 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:28.865163 (+ 87831us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.321230 (+456067us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:29.321269 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.321339 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.502925 (+181586us) raft_consensus.cc:1685] finished
0709 13:20:29.502946 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.503082 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27923,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":877803,"threads_started":2,"wal-append.queue_time_us":553663}
I20260709 13:20:29.520838 22973 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:20:29.556378 16591 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 3211) took 1735 ms (client timeout 2003 ms). Trace:
W20260709 13:20:29.619467 16591 rpcz_store.cc:269] 0709 13:20:27.820704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.450688 (+629984us) service_pool.cc:224] Handling call
0709 13:20:29.556182 (+1105494us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400235fd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:29.556324 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710063360}
I20260709 13:20:29.619882 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.621526 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.624363 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.624502 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.624949 16591 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:29.625088 22917 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.625821 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.627542 16668 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:29.557154 22930 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.558418 22947 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:29.559923 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 3494) took 1260 ms. Trace:
W20260709 13:20:29.560675 22758 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:20:29.564805 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.520730 22906 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:29.629689 16663 rpcz_store.cc:276] 0709 13:20:28.299560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.299673 (+   113us) service_pool.cc:224] Handling call
0709 13:20:29.559731 (+1260058us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b5400145854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:29.559843 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187246464}
I20260709 13:20:29.630571 22906 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.629930 22758 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.631578 22930 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.632766 22906 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.632992 22758 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.629576 22917 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.567934 22933 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:29.568259 22983 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.636210 22983 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.578977 16710 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.581141 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:29.557224 22865 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.633448 17416 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
W20260709 13:20:29.567265 16736 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 2546) took 1841 ms (client timeout 1966 ms). Trace:
W20260709 13:20:29.638543 16736 rpcz_store.cc:269] 0709 13:20:27.725662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.064801 (+339139us) service_pool.cc:224] Handling call
0709 13:20:29.567088 (+1502287us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b5400306954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:29.567193 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512840448}
I20260709 13:20:29.640233 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.640233 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.651032 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.652238 16588 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:29.657687 22707 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:29.658749 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 82588.9us
I20260709 13:20:29.659650 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.664117 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.666011 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.667029 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.667506 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.670150 22707 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:29.671432 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.667629 16667 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:29.673125 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.664942 16663 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:29.575623 22777 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:29.682083 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 152517us
I20260709 13:20:29.575462 22969 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:29.575066 22980 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:29.688938 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.657377 22786 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:29.658994 22962 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:29.659035 22868 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:29.695812 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.695911 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:29.705175 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 126287us
I20260709 13:20:29.665838 22692 raft_consensus.cc:1081] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:29.706247 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.708106 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:29.674937 22983 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.709067 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.709374 16737 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:29.710610 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:29.711195 16671 raft_consensus.cc:606] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.712523 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.712855 22943 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:29.713101 16559 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:29.713373 22943 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:29.713919 22917 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:29.714162 22895 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:29.714735 22933 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:29.725360 16665 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:20:29.715178 22906 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:29.717357 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.718219 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.737994 16743 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:29.738467 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.739048 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.739683 16743 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:29.718899 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.713610 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.720489 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:29.721367 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.741209 16741 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 9.
I20260709 13:20:29.689666 22777 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:29.747038 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.689137 22937 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:29.689518 22947 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:20:29.730783 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 141234us
W20260709 13:20:29.731789 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 323728us
I20260709 13:20:29.696578 22980 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.574837 22910 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.741887 22707 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:29.739231 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:29.742074 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.742892 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.743642 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.744376 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.787986 22707 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.745702 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.741386 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:29.789057 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47733.6us
I20260709 13:20:29.750856 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.753950 22833 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.755366 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.757135 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.714303 22943 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.723515 22983 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.723220 22917 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.748987 22906 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:29.783506 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.792927 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:29.840271 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56846.3us
I20260709 13:20:29.788656 22707 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:29.787609 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48460.1us
I20260709 13:20:29.798705 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:20:29.800072 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 176300us
I20260709 13:20:29.788946 16742 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:29.816533 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.827136 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.843104 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.782552 16587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
W20260709 13:20:29.801643 22993 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:20:29.832518 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:29.819664 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:29.838100 16745 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:29.775465 22947 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.857828 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.804462 22909 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.801437 22980 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:29.864336 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.853356 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:29.856060 16588 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:29.867341 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:29.856802 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.844028 16587 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:29.868515 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:29.837286 22777 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:29.862511 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 63913.6us
I20260709 13:20:29.862812 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:29.868908 16587 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:29.860097 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.869634 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.832208 22917 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:29.881336 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.880821 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:29.881649 22869 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:29.882014 22856 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:29.881964 22782 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:29.882457 22871 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:29.899037 22871 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.899525 22871 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:29.862619 22909 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.892796 22856 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.896727 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:29.896345 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:29.850201 22793 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:29.899180 22984 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:20:29.902853 22909 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.903342 22856 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:29.910306 16669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3511) took 1207 ms. Trace:
I20260709 13:20:29.911593 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:29.903473 22991 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:20:29.903995 16663 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:29.903635 22692 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:20:29.916813 16634 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:29.904764 22973 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:29.912648 22871 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.905647 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:29.920547 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.921762 22833 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.921890 22871 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:29.923426 22895 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:29.884883 22862 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:29.927135 22970 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:29.913833 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.927755 22970 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.911970 16669 rpcz_store.cc:276] 0709 13:20:28.702940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.766479 (+ 63539us) service_pool.cc:224] Handling call
0709 13:20:28.867850 (+101371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.311377 (+443527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.311407 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:29.329297 (+ 17890us) log.cc:844] Serialized 26 byte log entry
0709 13:20:29.587240 (+257943us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:29.587278 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:29.587410 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:29.910094 (+322684us) raft_consensus.cc:1685] finished
0709 13:20:29.910113 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:29.910252 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1274,"raft.queue_time_us":54361,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6810,"thread_start_us":621181,"threads_started":2,"wal-append.queue_time_us":396849}
I20260709 13:20:29.928203 22970 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:29.928769 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.858474 22929 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:29.927074 22895 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.896533 22872 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:20:29.863096 22947 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:29.930796 22895 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:29.884744 22889 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:29.927533 22862 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:29.943033 22862 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:29.944648 22957 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.933954 16737 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:29.932665 22972 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:29.935309 16587 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.938710 22889 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:29.941110 22851 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:29.865240 22996 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:30.002794 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 79333.9us
I20260709 13:20:30.044734 16739 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:29.949630 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:30.055459 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:30.055836 16743 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:29.968788 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:30.058616 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109075us
I20260709 13:20:29.977455 22862 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.054729 16668 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index 10 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:29.965751 22856 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.007475 22992 raft_consensus.cc:1064] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:29.970234 22895 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.065994 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.975138 22909 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:29.966542 22947 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.067827 16666 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:30.068396 22947 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.994992 22899 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index 10 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:30.071012 22786 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:30.071249 22909 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:29.946818 22957 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.063321 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.972960 22962 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:29.890879 22869 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:30.074090 22962 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:30.074293 22869 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:29.959048 22929 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:30.077875 22929 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:30.072456 22695 consensus_queue.cc:1059] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:29.969336 22917 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.084354 22695 raft_consensus.cc:3055] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:30.084739 22695 raft_consensus.cc:740] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:20:30.085258 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 116584us
I20260709 13:20:30.085575 22695 consensus_queue.cc:260] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.086509 22917 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.016170 22970 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.087368 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:30.020488 22906 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.081216 22937 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:30.063494 22862 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:30.000021 22983 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.071463 22786 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:29.966275 16489 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 13 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:30.096781 22786 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:30.082731 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:30.084509 16587 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.062987 22889 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.066272 22895 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.087909 16669 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:20:30.096796 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:30.101460 16737 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:30.101960 22937 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:30.103250 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.103168 22983 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:30.103999 22983 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:30.104368 22983 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:30.104738 22889 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.088155 22972 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:30.088748 22695 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:30.082607 22962 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.111797 22962 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.089502 22970 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.074061 22851 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:30.017110 22777 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.090214 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:30.091380 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:29.927198 22933 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.152s
I20260709 13:20:29.960156 22791 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:30.061861 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:30.116189 16669 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:30.118252 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:20:30.122702 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60905.8us
I20260709 13:20:30.122797 22970 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:30.066208 22992 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.149425 16669 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:20:30.153435 16587 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:20:30.153859 16587 pending_rounds.cc:85] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 10
I20260709 13:20:30.154060 16587 pending_rounds.cc:107] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:20:30.154271 16587 raft_consensus.cc:2889] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:30.155589 16590 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:30.155937 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:30.158977 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.159416 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:30.159950 16590 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:30.160516 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.162351 16590 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:30.162940 22999 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:30.163493 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.164352 16587 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:30.114452 16737 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:30.096800 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:30.169205 16739 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:30.134382 22862 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:30.170186 22917 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:30.136878 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.140435 16591 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:20:30.141002 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.141525 22851 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:30.172201 22851 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:30.172595 22851 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:30.175154 16592 raft_consensus.cc:2393] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:30.129072 22983 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.147349 22970 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.126631 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:30.204510 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 77988.1us
I20260709 13:20:30.204599 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.128150 22791 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.284162 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.083627 22856 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:30.093380 22969 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.121415 16666 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:30.121642 16665 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:30.286470 16665 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:30.284171 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 399092us
I20260709 13:20:30.099239 16489 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:30.010169 22980 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.130929 22707 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:30.299788 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 132919us
W20260709 13:20:30.221604 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.293103 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:30.294016 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:30.295462 16667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.039s	sys 0.114s
W20260709 13:20:30.306033 16667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.039s	sys 0.114s
I20260709 13:20:30.296205 16489 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:30.312546 22989 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:30.182958 22917 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:30.308691 16665 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:30.317116 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.122125 22933 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:30.313840 22989 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:30.283200 22970 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:30.342929 16736 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 20: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:30.342736 22537 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:30.343756 22707 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.341370 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.301637 22692 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:30.356361 22853 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:30.356803 22853 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:30.357623 22853 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.362474 16666 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:30.355295 22537 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:20:30.373175 16665 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:30.380913 22869 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.336347 22792 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:30.314584 22868 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:20:30.386958 22982 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:30.353935 22786 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.391423 16596 raft_consensus.cc:2410] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:20:30.351801 22692 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:30.402040 22991 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:30.369354 22853 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:30.325014 22933 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:30.396436 22537 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:30.338814 22864 consensus_queue.cc:1059] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:30.404490 22991 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:30.392918 22980 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:30.409840 16741 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
W20260709 13:20:30.414316 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.005s	sys 0.009s
W20260709 13:20:30.415963 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.005s	sys 0.009s
W20260709 13:20:30.416289 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 530676us
W20260709 13:20:30.410404 22943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.020s	sys 0.020s
I20260709 13:20:30.405185 22692 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:30.394588 22856 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
W20260709 13:20:30.419637 22943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.021s	sys 0.021s
W20260709 13:20:30.446384 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.420617 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.009s	sys 0.004s
W20260709 13:20:30.450341 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.004s
I20260709 13:20:30.453962 16666 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:30.420377 22868 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:20:30.420735 22968 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.010s	sys 0.010s
W20260709 13:20:30.450727 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 575004us
I20260709 13:20:30.400754 22917 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.463145 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.469285 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.470132 22537 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:30.459013 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:30.463930 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.465207 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.472651 16594 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:20:30.473433 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.465674 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:30.471618 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 333939us
I20260709 13:20:30.474442 16587 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:30.475841 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.475276 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:30.477780 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:30.478312 17409 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.023s	sys 0.042s
I20260709 13:20:30.469515 16595 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:30.478122 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:30.478632 17409 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.023s	sys 0.042s
I20260709 13:20:30.479354 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.478823 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:30.473953 16594 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:20:30.481004 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.475315 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:30.474767 17378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.029s	sys 0.025s
W20260709 13:20:30.483564 17378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.029s	sys 0.025s
I20260709 13:20:30.484791 22777 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:30.486006 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.499035 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.499948 16592 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
W20260709 13:20:30.471797 22968 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.011s	sys 0.010s
I20260709 13:20:30.506260 16591 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:30.482822 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.518044 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:30.518253 16590 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:20:30.487569 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:30.511442 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.514832 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:30.519718 17374 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.523641 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:30.525728 22707 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:30.526183 22707 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:30.526285 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:30.526562 22707 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:30.531414 17411 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:30.532953 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.544200 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:30.544824 17420 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:30.545486 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:30.460891 22868 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:30.548417 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:30.510097 22899 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:42550 (request call id 18) took 4203 ms. Trace:
I20260709 13:20:30.483970 22926 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:30.504772 22943 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:30.564167 22707 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.565080 17419 raft_consensus.cc:1813] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Handling vote request from an unknown peer efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.565470 17419 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:30.568651 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.569319 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:30.573717 22983 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.543448 22788 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:30.543285 22917 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:30.556872 22937 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:30.483695 22862 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:20:30.551818 23004 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:30.456537 22991 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.519093 23006 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:20:30.394429 22953 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.176s
I20260709 13:20:30.586311 17420 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:30.557942 22889 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:30.602487 22707 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.603559 16665 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:30.614931 22537 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:30.574360 22947 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:30.571641 22969 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:30.641077 22537 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.471264 22933 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.650398 22933 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:30.652102 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1291) took 1055 ms. Trace:
I20260709 13:20:30.652526 17409 rpcz_store.cc:276] 0709 13:20:29.596744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.598250 (+  1506us) service_pool.cc:224] Handling call
0709 13:20:29.598543 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:29.668408 (+ 69865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:29.668437 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:29.683840 (+ 15403us) log.cc:844] Serialized 26 byte log entry
0709 13:20:30.479749 (+795909us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:30.479803 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:30.479888 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:30.651875 (+171987us) raft_consensus.cc:1685] finished
0709 13:20:30.651898 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:30.652034 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64067,"raft.queue_time_us":76747,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":21540,"thread_start_us":755402,"threads_started":1,"wal-append.queue_time_us":945252}
W20260709 13:20:30.666256 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.669541 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:30.670433 17414 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:30.602846 22791 raft_consensus.cc:3055] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:30.575167 22957 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:30.592556 22962 raft_consensus.cc:993] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:30.649102 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:30.692783 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43799.4us
W20260709 13:20:30.694224 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.587802 22983 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:30.609458 22793 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:30.682416 16665 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:20:30.739145 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.749100 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.740270 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 377107us
I20260709 13:20:30.624590 23004 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:30.626755 22889 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:30.751096 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:30.623402 23005 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:30.756650 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 415329us
I20260709 13:20:30.685649 22786 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:30.695820 22991 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:30.757354 23013 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:20:30.763938 22889 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:30.764199 23005 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:30.752617 22793 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:30.797724 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.798264 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.821179 16591 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:30.819846 16636 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:30.820717 17414 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:30.822592 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:30.751488 22957 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.799798 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:30.826974 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 299516us
I20260709 13:20:30.827410 16587 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:30.696151 22864 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:30.803216 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:30.763470 22792 raft_consensus.cc:1081] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
W20260709 13:20:30.836663 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 288120us
I20260709 13:20:30.751478 22869 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.753412 22992 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:30.839563 22707 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:30.764659 22788 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:20:30.821233 16590 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:30.799083 16592 raft_consensus.cc:1240] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:30.757702 23004 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:30.799363 22537 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:30.675962 22868 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.779912 22969 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:30.754487 22983 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.840130 22707 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:30.686590 22791 raft_consensus.cc:740] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:30.842053 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:30.878226 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:30.835845 22864 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:30.887141 16590 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
W20260709 13:20:30.876471 22851 consensus_meta.cc:287] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.008s
I20260709 13:20:30.820384 23005 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:30.843325 16587 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:20:30.883217 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 307086us
I20260709 13:20:30.895802 17419 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:20:30.872242 22909 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:30.897048 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:30.871970 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 328361us
I20260709 13:20:30.898288 22707 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:30.898813 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:30.899924 16591 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:30.902182 23016 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 13:20:30.558555 22899 rpcz_store.cc:276] 0709 13:20:26.284437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:26.291991 (+  7554us) service_pool.cc:224] Handling call
0709 13:20:26.357077 (+ 65086us) log.cc:844] Serialized 487 byte log entry
0709 13:20:30.483292 (+4126215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41544,"raft.queue_time_us":42533,"raft.run_cpu_time_us":1082,"raft.run_wall_time_us":1082,"thread_start_us":78412,"threads_started":1,"wal-append.queue_time_us":133977,"wal-append.run_cpu_time_us":1526,"wal-append.run_wall_time_us":1003928}
I20260709 13:20:30.869589 22992 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:30.881093 22962 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:30.906952 17420 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:30.408279 22982 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:30.890424 22868 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:30.919669 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:20:30.904233 23002 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 13:20:30.867237 22685 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:20:30.922519 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:30.923830 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:30.911379 22929 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.824s	user 0.009s	sys 0.003s
I20260709 13:20:30.924220 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:30.924712 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:30.926668 17414 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:30.904174 22873 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:30.903357 22943 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:30.933166 16669 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 10: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:30.900800 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.915269 22909 raft_consensus.cc:697] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:30.914083 22962 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:31.000280 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:31.001416 16662 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:30.899554 22851 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:30.909485 22869 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
W20260709 13:20:31.012421 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 189901us
I20260709 13:20:31.022074 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:31.022796 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.023039 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.023717 16743 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:30.932854 22873 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:30.935871 22943 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:31.029242 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 128550us
I20260709 13:20:31.031549 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.033512 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:31.034536 16737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:31.035133 16737 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:31.035516 16737 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:31.048169 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.048841 17416 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:31.049273 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.000185 22926 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:31.053085 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:31.053046 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 134965us
I20260709 13:20:31.055099 16740 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:31.060271 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.008499 22962 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:31.059820 16742 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:31.060918 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.067320 16739 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.059588 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.096004 16743 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
W20260709 13:20:31.115388 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.058377 16737 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.132697 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:30.932073 22791 consensus_queue.cc:260] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.134469 22791 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:31.135579 17411 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:31.047492 22873 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:31.060320 23011 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:31.143318 23011 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:31.143728 23011 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:20:31.147303 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 86037.3us
I20260709 13:20:31.147360 16741 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:31.147912 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:30.934758 22929 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.155843 16745 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 10: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.156402 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.152654 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.153427 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.155027 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.159500 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.160991 16737 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.151405 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.170542 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.173502 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.047839 22872 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:31.177084 16662 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:20:31.168337 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:31.180766 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:31.186452 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.542s (SENT)
I20260709 13:20:31.191313 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:31.060379 22910 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:31.176638 23002 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 13:20:31.206132 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:31.182164 22862 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:31.201069 22872 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:31.311514 23030 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:20:31.312863 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.210076 22786 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:31.314654 22786 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.317582 22786 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:31.230201 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.048951 23019 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.165s
W20260709 13:20:31.334744 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 154068us
I20260709 13:20:31.326650 22851 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:31.183025 22972 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:31.168967 23022 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:20:31.342239 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 186627us
W20260709 13:20:31.342670 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 172231us
I20260709 13:20:31.175405 23029 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:20:31.198506 23012 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.398s	user 0.001s	sys 0.000s
I20260709 13:20:31.346150 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.348647 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:31.334779 22853 consensus_meta.cc:287] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.921s	user 0.008s	sys 0.003s
I20260709 13:20:31.193413 22988 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:31.339983 22970 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.979s	user 0.009s	sys 0.003s
W20260709 13:20:31.351495 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 145469us
I20260709 13:20:31.351598 22537 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:31.067171 22992 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.357234 22537 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.301048 22929 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:31.191370 22957 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.343072 16710 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:31.345230 22917 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.001s
I20260709 13:20:31.200091 22910 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:31.346884 23029 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:31.358729 23018 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:31.326552 22793 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.347538 22953 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:31.366312 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:31.362665 23035 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:31.369297 16594 raft_consensus.cc:2410] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:20:31.369419 22537 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.371340 23029 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:31.357615 22970 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.388590 22926 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:31.390331 22970 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.357957 22983 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.355077 22685 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:20:31.392554 22989 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:31.393023 22989 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:31.394662 22933 consensus_meta.cc:287] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.744s	user 0.009s	sys 0.003s
I20260709 13:20:31.383045 22910 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:31.374231 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.388103 23035 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:31.405109 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.406088 22926 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:31.407021 22685 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:31.407503 22777 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:31.407986 22777 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:31.407423 22926 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.408793 22777 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.409751 22777 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:31.411273 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:31.412065 17414 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 20: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:31.405334 23035 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:31.371142 22929 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:31.429329 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.422983 16669 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:20:31.426971 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.197382 22909 consensus_queue.cc:237] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.442523 17374 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.447110 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.447516 22962 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:31.448484 22692 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 1.025s	user 0.010s	sys 0.004s
I20260709 13:20:31.449048 22692 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.450039 23029 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.407431 22685 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.437633 23011 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.454399 23011 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.433245 22929 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:31.456099 22929 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:31.414832 23008 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.611s	user 0.006s	sys 0.011s
I20260709 13:20:31.459007 16666 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:31.433871 22782 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:31.460704 22782 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.461076 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.449572 22962 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.459987 16665 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:31.451694 22685 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.386915 22917 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.092s
I20260709 13:20:31.386300 22953 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:31.462779 22782 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.490139 16738 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 19. Current term is 20. Ops: []
W20260709 13:20:31.490852 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.415021 22989 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.473803 22968 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.491703 22989 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.491808 22968 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:31.493806 22970 consensus_queue.cc:1175] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 14 status { last_received { term: 14 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:31.478646 22853 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:20:31.495905 22685 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:31.496317 22853 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:31.497709 22853 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:20:31.421626 22889 consensus_meta.cc:287] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.595s	user 0.007s	sys 0.002s
I20260709 13:20:31.498726 22889 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.498057 22853 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.499958 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:31.502104 16665 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:31.485785 16745 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:20:31.501348 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57335.6us
I20260709 13:20:31.482357 16487 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 231) took 3262 ms. Trace:
I20260709 13:20:31.487385 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.507140 17409 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:31.519937 16741 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.464910 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.520982 16737 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:31.521710 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 157777us
I20260709 13:20:31.423673 22793 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:31.465310 23031 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 13:20:31.462723 23004 consensus_meta.cc:287] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.585s	user 0.005s	sys 0.003s
I20260709 13:20:31.423532 22982 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.530069 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:31.509564 23005 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.002s
I20260709 13:20:31.488036 23019 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:31.534626 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.534997 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:31.476792 22969 consensus_meta.cc:287] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.000s
I20260709 13:20:31.524222 22957 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:31.503901 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.571339 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 3241) took 1103 ms. Trace:
I20260709 13:20:31.538787 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:31.571943 16594 rpcz_store.cc:276] 0709 13:20:30.467595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:30.480778 (+ 13183us) service_pool.cc:224] Handling call
0709 13:20:31.571068 (+1090290us) spinlock_profiling.cc:243] Waited 887 ms on lock 0x7b540010ed54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:31.571255 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2040611584}
I20260709 13:20:31.529207 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.495208 22968 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.500902 22953 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:31.468456 22692 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:31.563326 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 370973us
I20260709 13:20:31.506789 22889 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.575258 16587 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:31.505004 16487 rpcz_store.cc:276] 0709 13:20:28.220067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:28.220229 (+   162us) service_pool.cc:224] Handling call
0709 13:20:31.482221 (+3261992us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:29.503303 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:29.503869 (+   566us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:29.503899 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:20:29.503917 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:29.504231 (+   314us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:29.504297 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:29.504322 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:29.504504 (+   182us) tablet.cc:701] Row locks acquired
0709 13:20:29.504518 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:20:29.504654 (+   136us) write_op.cc:270] Start()
0709 13:20:29.504760 (+   106us) write_op.cc:276] Timestamp: P: 1783603229504608 usec, L: 0
0709 13:20:29.504780 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:20:29.505437 (+   657us) log.cc:844] Serialized 530 byte log entry
0709 13:20:30.614255 (+1108818us) op_driver.cc:464] REPLICATION: finished
0709 13:20:31.067563 (+453308us) write_op.cc:301] APPLY: starting
0709 13:20:31.140009 (+ 72446us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:31.141270 (+  1261us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:31.141378 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:31.142166 (+   788us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:31.142380 (+   214us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:31.142431 (+    51us) write_op.cc:312] APPLY: finished
0709 13:20:31.142645 (+   214us) log.cc:844] Serialized 25 byte log entry
0709 13:20:31.142829 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:31.156023 (+ 13194us) write_op.cc:454] Released schema lock
0709 13:20:31.169968 (+ 13945us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:31.325440 (+155472us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:31.325471 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":400109,"apply.run_cpu_time_us":3858,"apply.run_wall_time_us":163145,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"num_ops":1,"prepare.queue_time_us":452960,"prepare.run_cpu_time_us":6458,"prepare.run_wall_time_us":124040,"raft.queue_time_us":663861,"replication_time_us":1109262,"spinlock_wait_cycles":6429952,"thread_start_us":1249743,"threads_started":4,"wal-append.queue_time_us":227695}]]}
I20260709 13:20:31.582772 16737 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
W20260709 13:20:31.587718 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 213607us
I20260709 13:20:31.607141 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:31.481711 22791 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.082s
I20260709 13:20:31.608585 22791 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:20:31.598258 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 151379us
I20260709 13:20:31.477962 22873 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:31.569000 22991 consensus_meta.cc:287] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.765s	user 0.009s	sys 0.003s
I20260709 13:20:31.612479 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.513026 23035 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:31.587657 22868 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.660s	user 0.007s	sys 0.003s
I20260709 13:20:31.623296 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.625134 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.626158 16594 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:31.615103 16666 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 13:20:31.616194 16587 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:31.616286 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:31.628301 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.613900 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.559121 22792 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:31.589278 22969 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:31.608940 22791 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.637239 16588 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:31.631987 16670 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.603466 22906 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:31.629150 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:31.633998 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.633811 22537 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:31.628434 22788 raft_consensus.cc:2764] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:20:31.630046 16559 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:31.642292 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:31.609449 23038 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:31.644114 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.630692 22889 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:31.540226 22864 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.667305 23038 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:31.669992 22943 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.001s
I20260709 13:20:31.670138 23038 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.632978 22695 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:31.635200 23024 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:20:31.632598 22692 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:31.646276 22707 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:31.634974 22991 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:31.553143 23004 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.673377 22791 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.674628 22791 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:31.675129 17415 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:20:31.647603 23041 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.065s	user 0.002s	sys 0.002s
I20260709 13:20:31.647900 23029 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.649839 23011 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:31.636693 22792 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.678522 22869 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:31.679394 22792 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.645644 22992 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:31.680040 22992 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:31.680442 22992 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:31.652467 16670 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:31.659930 22873 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.660619 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.617558 22853 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.666235 23035 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.559592 23005 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:31.697945 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55762.2us
I20260709 13:20:31.683673 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:31.684326 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 180533us
I20260709 13:20:31.699646 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:31.684751 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 116549us
I20260709 13:20:31.691066 22707 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:31.699448 16593 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:31.700865 22707 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:31.629323 22957 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:20:31.674715 23004 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.707526 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.714895 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:31.715027 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73028.9us
I20260709 13:20:31.717499 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:31.717731 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:31.729183 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.731854 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.741338 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.742448 16737 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.743218 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.745849 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.749361 16671 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:31.747306 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.740048 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.748652 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:31.752331 16666 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:31.689100 22786 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:31.729804 22707 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.743264 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.755095 16744 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.757588 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.758369 16593 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.755703 23023 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.758880 23023 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.761372 23023 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:31.764659 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:31.766945 22853 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:31.640765 22929 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 13:20:31.770073 22929 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:31.770440 22929 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:31.755668 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.689285 22869 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:20:31.685408 23047 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:31.771548 22869 consensus_queue.cc:1243] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:20:31.771801 23047 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.753065 22982 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:31.772004 23032 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:31.772523 23032 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:31.772789 23047 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.773324 23032 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.774523 23032 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:31.775426 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.776656 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.777380 16592 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.777709 16587 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:31.778201 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.779969 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.780627 16590 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.781773 16591 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:20:31.782722 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.783288 16591 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:31.783169 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:31.784178 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:31.785074 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.785070 17419 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 14: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:31.779724 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.786774 16593 raft_consensus.cc:2393] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.734791 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.670845 22943 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.788863 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.790109 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.791874 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.792487 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:31.793447 16593 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:31.729022 23007 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:31.797868 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:31.809909 22953 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.812469 22953 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:31.815002 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.816579 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.817612 16591 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.818174 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.818179 22695 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:31.819024 22695 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:31.819415 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:31.820087 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.819823 22695 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:31.820539 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.820432 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.821569 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.823137 22695 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:31.825992 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.820102 16592 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:31.848521 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.849704 16740 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 20: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.852574 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.853399 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.854270 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.855306 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:31.788595 23016 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:20:31.866253 16743 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:31.857762 17419 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:31.851677 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.867866 16738 raft_consensus.cc:1240] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:31.859990 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.784211 23017 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:20:31.869540 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.869834 16737 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:20:31.865154 16666 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
W20260709 13:20:31.871275 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 136640us
I20260709 13:20:31.871944 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:31.872004 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:31.872828 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.865422 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.873503 16738 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:31.772539 22982 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.875439 22707 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:31.876250 16737 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:31.877564 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e8b0d179ad714b59b34ab1dc57a57c02 with cas_config_opid_index 10: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:20:31.877611 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.884003 16745 raft_consensus.cc:2393] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:31.885465 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:31.887367 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64316.7us
I20260709 13:20:31.753290 22947 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
W20260709 13:20:31.890915 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64273.2us
I20260709 13:20:31.891117 16741 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:20:31.891592 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:31.890847 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.894519 22537 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:31.896471 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.897471 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:31.896140 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:31.898170 16744 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:31.869218 23017 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.899010 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:31.900416 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.909025 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:31.896097 22933 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:31.889285 23050 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:31.909514 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.916203 16634 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:31.899447 22873 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.899703 22895 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:31.903805 22777 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:31.917776 16594 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:31.918330 22537 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:31.918583 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.918955 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.936465 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.937827 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:31.919440 23042 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:20:31.919525 16587 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:20:31.920522 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.921358 16666 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:31.925555 16740 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:31.929620 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.930806 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.931716 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.944447 16595 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:20:31.944875 23005 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:31.944994 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.945277 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.941972 22873 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.930771 16736 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:31.933157 16662 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:31.946432 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.901052 22685 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:20:31.940001 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.951933 22685 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:31.952739 22685 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:31.932425 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.954257 22685 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.947682 22873 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:31.947781 22909 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:31.955662 22685 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:31.948880 16593 raft_consensus.cc:2368] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:20:31.949728 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:31.957639 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:31.961485 16595 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:31.961982 16595 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:31.910197 22996 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:31.963096 16595 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.963887 23040 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:31.964164 16595 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:20:31.966480 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:31.966776 22856 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:20:31.967904 22856 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:31.968614 23024 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:31.968732 16670 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:31.969051 23024 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:31.970032 23024 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:31.974088 16670 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:31.952060 22969 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:31.963591 16592 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:31.975704 22969 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:31.977525 22983 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:31.978044 16594 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:20:31.978520 16594 pending_rounds.cc:85] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 7
I20260709 13:20:31.978776 16594 pending_rounds.cc:107] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:20:31.979038 16594 raft_consensus.cc:2889] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:31.983714 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.985019 22873 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:31.985927 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.986969 16587 raft_consensus.cc:2368] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16.
W20260709 13:20:31.991019 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.003419 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:31.940722 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:31.917912 22933 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:31.963830 22996 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:20:32.022624 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 81987.1us
I20260709 13:20:31.940874 22962 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:32.034796 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.036428 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:32.037292 16591 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:32.038693 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:32.039849 22537 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:32.040295 22537 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.040678 22537 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:32.041930 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 147719us
W20260709 13:20:32.042110 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 140440us
I20260709 13:20:32.024405 22933 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.046388 16664 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:32.025972 23024 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:32.043483 17376 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:32.025496 22793 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:32.021432 22982 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:32.048730 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.054575 17409 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:32.067615 22537 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.049180 23007 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:31.974679 23017 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.088451 22968 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:32.049417 22969 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.090255 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 333) took 2739 ms. Trace:
I20260709 13:20:32.055466 23011 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:32.097514 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.102643 16595 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:20:32.104207 16595 pending_rounds.cc:85] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 12
I20260709 13:20:32.105568 16595 pending_rounds.cc:107] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:20:32.106263 16595 raft_consensus.cc:2889] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:32.055305 23056 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:20:32.104919 23007 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:32.050314 22996 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.049815 22910 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.254848 16589 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: []
W20260709 13:20:32.260222 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 171292us
W20260709 13:20:32.138600 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:32.267321 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 99844us
I20260709 13:20:32.271176 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:32.272590 16592 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:32.096896 16489 rpcz_store.cc:276] 0709 13:20:29.350516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:29.350999 (+   483us) service_pool.cc:224] Handling call
0709 13:20:32.090188 (+2739189us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:30.896530 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:31.175277 (+278747us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:31.182025 (+  6748us) write_op.cc:435] Acquired schema lock
0709 13:20:31.191418 (+  9393us) tablet.cc:662] Decoding operations
0709 13:20:31.388188 (+196770us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:31.388288 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:31.388320 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:20:31.388713 (+   393us) tablet.cc:701] Row locks acquired
0709 13:20:31.388736 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:20:31.388907 (+   171us) write_op.cc:270] Start()
0709 13:20:31.389022 (+   115us) write_op.cc:276] Timestamp: P: 1783603231388853 usec, L: 0
0709 13:20:31.389047 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:20:31.389811 (+   764us) log.cc:844] Serialized 1294 byte log entry
0709 13:20:31.662664 (+272853us) op_driver.cc:464] REPLICATION: finished
0709 13:20:31.668067 (+  5403us) write_op.cc:301] APPLY: starting
0709 13:20:31.668153 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:31.670178 (+  2025us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:31.670289 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:31.681281 (+ 10992us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:31.681448 (+   167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:20:31.681492 (+    44us) write_op.cc:312] APPLY: finished
0709 13:20:31.681699 (+   207us) log.cc:844] Serialized 41 byte log entry
0709 13:20:31.689264 (+  7565us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:31.702739 (+ 13475us) write_op.cc:454] Released schema lock
0709 13:20:31.713542 (+ 10803us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:31.772982 (+ 59440us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:31.773013 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":5129,"apply.run_cpu_time_us":6107,"apply.run_wall_time_us":104863,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12552,"num_ops":3,"prepare.queue_time_us":552595,"prepare.run_cpu_time_us":4733,"prepare.run_wall_time_us":508147,"replication_time_us":273000,"thread_start_us":371776,"threads_started":1}]]}
I20260709 13:20:32.274156 16741 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:32.282223 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:32.109397 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60716.1us
I20260709 13:20:32.297477 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.118752 22868 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:32.304648 22868 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:32.154103 22929 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.110803 22777 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.197s
I20260709 13:20:32.190192 22793 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:32.308760 22933 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.162824 22864 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:32.173591 22943 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:32.201572 22962 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:32.205957 23017 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:20:32.155490 23019 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.006s
I20260709 13:20:32.209569 22906 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:32.334200 22906 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:20:32.334580 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 72943.5us
I20260709 13:20:32.335964 22906 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.336486 23023 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:32.157318 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:32.340029 17409 raft_consensus.cc:2393] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:32.164746 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.162693 22910 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.343005 22889 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:32.167315 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.343747 22980 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:32.344331 22980 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:32.147981 22537 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.343562 22889 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:32.347666 22786 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.347884 23019 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:32.348063 22889 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:32.348524 22786 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.346019 22980 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.353006 22980 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.119441 23007 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.121003 22992 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.361187 22992 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:32.131960 22996 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:32.310335 22793 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:32.308473 22868 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.328207 22685 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.332175 22929 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:32.332970 16592 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:32.333493 22933 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:32.154457 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:32.343384 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 178584us
I20260709 13:20:32.338656 22906 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:32.377720 16589 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:32.379287 16591 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:32.346547 22692 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.383759 16588 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:32.391266 22685 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.392761 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:32.393610 16671 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:32.395241 22943 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:32.396816 23004 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:32.397189 23004 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:32.398422 23004 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.400061 23004 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:32.409970 17420 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:20:32.402801 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:32.418620 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:32.419435 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52465.7us
W20260709 13:20:32.420151 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 123117us
I20260709 13:20:32.407233 22782 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:32.430536 22782 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.433420 22782 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:32.435216 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:32.436700 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63759.4us
I20260709 13:20:32.439296 22853 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:32.410019 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.455920 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:32.455942 22793 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.457060 22868 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:32.463392 16671 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:32.419242 22685 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.419122 22786 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:32.457051 22793 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:32.467519 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.473258 22793 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.473166 22889 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.474843 16743 raft_consensus.cc:1275] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:20:32.474891 23019 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:20:32.476840 22889 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:32.473766 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.475242 22910 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:20:32.479226 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:32.475410 16741 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:20:32.480134 16736 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:32.487016 22782 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:32.491349 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:32.492455 16743 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
W20260709 13:20:32.498302 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 88425us
I20260709 13:20:32.500921 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:20:32.506996 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.507588 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:20:32.509191 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 48123.1us
I20260709 13:20:32.510674 22537 consensus_queue.cc:1048] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:20:32.511372 22929 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:32.512197 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:32.515054 22685 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:32.419678 22692 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.521734 22537 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:32.522343 22537 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.522617 22537 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:32.521247 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:32.525205 22889 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:32.529950 22889 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:32.537074 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:32.539742 22864 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 13:20:32.547410 16559 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:32.533978 22929 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:32.519764 22786 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:20:32.554204 22910 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:32.573490 22910 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:32.574087 22692 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.574816 22910 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.557971 22707 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.560988 17419 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:20:32.576825 17419 pending_rounds.cc:85] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 11
I20260709 13:20:32.577176 17419 pending_rounds.cc:107] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:20:32.576908 22707 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.577462 17419 raft_consensus.cc:2889] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:32.577767 22943 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:32.578241 22943 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:32.564910 16736 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:32.579523 22707 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.547036 22980 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:32.579700 22943 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.580520 22980 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:32.581262 22943 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:32.583164 22980 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:32.568117 17374 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:32.583470 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 44892.9us
I20260709 13:20:32.584991 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.587806 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.589669 23017 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:32.590202 22957 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:32.590428 23017 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.591089 23017 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.595078 22782 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:32.598331 22929 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:32.601987 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.586462 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.604457 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:32.643859 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.647435 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:32.648474 16668 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:32.656688 23007 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:32.660512 16738 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:32.660734 16741 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:32.662981 22685 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:32.663367 22692 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:32.665233 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:32.665537 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 61987.8us
I20260709 13:20:32.666447 17374 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.668534 23048 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:32.668673 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.669405 23048 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:32.669806 23048 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:32.670135 23048 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:32.634464 22537 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:32.680074 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.689927 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:32.675091 22793 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:32.696792 22537 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:32.693830 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:32.700961 16738 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:32.703189 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:32.703903 16742 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:32.693327 22980 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:32.676862 23004 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:20:32.705742 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:32.723479 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 137204us
I20260709 13:20:32.705931 22980 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.739208 23023 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.739558 22980 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.740044 23023 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.742468 23023 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.743184 22996 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:32.730777 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 186648us
I20260709 13:20:32.751458 22957 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:32.731401 22786 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.752580 22786 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.734081 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:32.734476 22868 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:32.756114 22868 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:32.755820 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:32.757182 22868 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.755792 22906 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:32.730062 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.759907 22906 consensus_queue.cc:1243] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:20:32.760603 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44503.3us
I20260709 13:20:32.761229 22996 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:32.761515 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:32.763157 22947 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.764871 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.757218 16738 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:32.768165 16595 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:20:32.768250 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:32.768929 16595 pending_rounds.cc:85] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 9
I20260709 13:20:32.769345 16595 pending_rounds.cc:107] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 13.10
I20260709 13:20:32.769762 22853 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:32.770423 22864 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.771139 22864 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.771381 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.772073 17411 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:32.773342 22864 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.760597 22906 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:32.774649 22906 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:32.775931 22906 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.780465 16489 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 14 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.769861 16595 raft_consensus.cc:2917] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Aborting config change with OpId 13.10: Aborted: Op aborted by new leader
I20260709 13:20:32.770412 22853 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.794312 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:32.802326 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:32.816841 16742 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:32.817157 22786 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:32.817826 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53104.2us
I20260709 13:20:32.818233 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:32.819070 16591 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:32.820155 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:32.820612 22889 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:32.823972 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:32.831569 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 68687.9us
I20260709 13:20:32.832106 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:32.832536 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.834232 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:32.835007 16595 raft_consensus.cc:2889] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:32.835124 22947 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:32.803622 22853 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.840131 23034 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:32.840970 23034 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.842615 22937 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:32.843272 22937 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.813995 22777 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:32.847754 22777 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.848560 22937 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:32.803031 22933 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:32.850366 22777 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.850674 22782 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:32.851210 22782 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:32.852249 22782 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.853665 22782 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:32.853125 16489 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 14 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:32.856233 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:32.857306 22864 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:32.856083 23040 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:20:32.857770 22906 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.864161 22864 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:32.865351 22864 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:32.855427 16489 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 13 to 14, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 14 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:32.889276 22853 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.889580 22980 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.890575 16668 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:32.891064 16745 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:20:32.866884 23034 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:32.867215 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:32.875415 22889 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:32.886112 23040 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:32.886842 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52716.7us
W20260709 13:20:32.891824 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49162.2us
W20260709 13:20:32.892539 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.893020 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:32.922375 23040 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:32.923843 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:32.930716 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.940395 23040 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.940843 22853 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:32.941030 23017 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:32.941576 23084 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:20:32.942709 17411 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:32.943219 23017 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:32.943699 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:32.934666 22786 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:32.975660 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:32.978137 16741 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:32.988554 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.972458 22786 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:32.991139 22996 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:32.993822 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 161807us
I20260709 13:20:32.986833 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.991153 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.996758 17409 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:33.006572 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:32.995858 22786 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:33.018788 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 265563us
I20260709 13:20:33.022233 17376 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:20:33.022541 17376 leader_election.cc:403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:20:33.023116 23086 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.034577 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.035509 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:33.037050 22707 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.035501 16589 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:33.047752 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.035563 22926 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.056242 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.058729 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:33.059504 16745 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.062382 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:33.063562 22537 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:33.064046 22537 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.064435 22537 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.065143 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.068477 16738 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:20:33.069885 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:33.070590 16736 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:33.072417 23087 log.cc:927] Time spent T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:20:33.072979 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.075817 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:33.077731 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.078022 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.078143 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:33.078599 16593 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 17: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:33.080303 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.082670 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.083487 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.068928 16738 pending_rounds.cc:85] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 9
I20260709 13:20:33.087276 16738 pending_rounds.cc:107] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:20:33.087623 16738 raft_consensus.cc:2889] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:33.089047 16742 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:33.100966 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:33.101653 16669 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:33.100647 17412 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:33.103859 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.105020 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.105594 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.110322 16592 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:33.112054 16669 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:33.118889 22980 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.122045 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:33.151847 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:33.159965 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 145539us
I20260709 13:20:33.189292 16669 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 20. Ops: []
I20260709 13:20:33.229432 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.230432 16588 raft_consensus.cc:2376] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17. Re-sending same reply.
I20260709 13:20:33.085090 23048 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.240751 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.235215 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:33.244874 22537 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.220163 22788 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:33.252116 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:33.254527 17411 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:33.259140 17410 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:33.262125 17419 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:33.220749 22957 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:33.231755 22853 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:33.217919 22906 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:20:33.276504 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.277521 22537 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.278728 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.281803 16589 raft_consensus.cc:2393] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 9: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:33.288873 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:33.289583 16595 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.268003 22777 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:33.295756 17411 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:20:33.300284 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.304014 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.310084 16741 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.311739 22889 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.312536 17415 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:33.289495 22853 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.313620 17415 raft_consensus.cc:1217] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 12.12->[14.13-14.13]   Dedup: 14.13->[]
I20260709 13:20:33.314114 17415 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:20:33.314999 22853 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.315274 17415 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 21. Ops: []
I20260709 13:20:33.316211 17411 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:33.317536 17411 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:33.310561 17413 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:33.263495 22788 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.327706 16596 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:33.329025 17409 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:33.334838 16595 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.338286 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:33.340022 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 126010us
I20260709 13:20:33.343433 22788 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.344336 22788 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:33.344873 22788 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.347235 22788 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.348986 22788 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:33.337602 17414 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 LEARNER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:33.317418 22926 raft_consensus.cc:2764] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:20:33.316670 23048 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.352972 22537 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:33.353133 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:33.292769 22957 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.366107 17412 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9.
I20260709 13:20:33.367036 23017 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:33.366751 22873 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:20:33.361900 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.381508 16738 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 20. Ops: []
I20260709 13:20:33.363512 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:33.375578 23104 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:20:33.378350 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:33.381726 16587 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:33.335041 22793 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:33.390187 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.395905 17409 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:33.392154 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.401008 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:33.402259 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.403191 22873 consensus_queue.cc:1243] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:20:33.403728 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.404507 17415 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:33.405484 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.407794 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.408526 17412 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.413103 22910 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:33.413730 22910 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:33.403774 22873 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:33.415537 22793 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.415630 22873 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:33.384418 23017 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:33.418179 22873 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.419813 23017 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.422106 22957 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.423790 22873 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:20:33.418171 22793 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.416035 22910 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.406160 17415 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:33.427471 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:33.394793 23086 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:20:33.398305 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:20:33.438674 23098 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:20:33.438885 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:33.466174 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:33.442425 23034 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:33.467192 22910 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:33.447832 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.453552 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.455503 17411 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 13 LEARNER]: Advancing to term 14
W20260709 13:20:33.468871 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 144017us
I20260709 13:20:33.452843 17419 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:33.471513 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.472527 22906 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:33.473940 22906 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:20:33.477417 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 153137us
I20260709 13:20:33.478197 22692 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:33.479382 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:33.479857 22692 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.480387 22692 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.480764 22962 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.481269 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.481482 22962 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.482509 22692 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.482666 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.483915 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.488147 16736 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:33.489272 22962 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.497555 17420 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:33.506034 16739 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:33.399118 17416 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:33.497804 17412 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.515448 23021 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.517418 16745 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.518900 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.522681 23021 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:33.525313 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59276us
I20260709 13:20:33.526126 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:33.526258 23021 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.527043 22777 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:20:33.527676 16708 leader_election.cc:400] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:20:33.528054 16708 leader_election.cc:403] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:33.528849 22777 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:20:33.529208 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:33.530066 22777 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:33.531526 23048 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.532220 23048 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.534756 23048 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:33.536095 22869 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:20:33.537343 22869 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:33.537741 22869 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:33.539656 17419 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:33.541576 22786 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.520740 22926 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:20:33.545132 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.546231 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.546852 17409 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.530747 16708 leader_election.cc:400] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16.
I20260709 13:20:33.547981 16708 leader_election.cc:403] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:33.505713 22929 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.551369 16745 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:33.556623 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.515594 22980 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:20:33.565308 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:33.559706 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:33.556365 23111 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:33.551426 16561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.4:41455 timed out after 1.507s (SENT) [suppressed 31 similar messages]
I20260709 13:20:33.580334 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:33.556705 22926 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16.
I20260709 13:20:33.557924 22906 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.588076 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:33.594204 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.594823 16743 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.558944 23048 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:20:33.597004 16744 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:33.597704 16670 raft_consensus.cc:2368] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:20:33.603735 23048 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:33.604108 23048 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:33.604285 16664 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:33.605036 23048 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.586031 22943 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.607231 22943 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.609412 22943 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.586112 23004 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:33.611383 22786 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.613312 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:33.613925 16590 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:33.578909 23040 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:33.592094 23111 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.618364 23111 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.619088 22692 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.619623 22692 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.620981 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:33.621570 16671 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:20:33.621603 22692 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.622815 17411 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:20:33.622934 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.623648 16742 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:33.606622 23048 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:33.538664 22869 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.629091 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.633224 16591 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:33.628006 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:33.627311 16590 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:33.633628 16594 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:33.640770 16634 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:33.642320 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 126688us
I20260709 13:20:33.644737 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:33.648085 17409 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.650396 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.643097 16634 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:33.627876 23040 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:33.654284 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 123693us
I20260709 13:20:33.638676 22873 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:33.657788 22873 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:33.658250 22873 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.633384 22906 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.660591 22906 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.662112 22864 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.650709 22943 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.662739 22864 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.663316 22943 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.664090 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.664691 16743 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:33.665032 22864 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.665306 22943 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:33.657114 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.668733 16743 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:33.653622 22869 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:33.669903 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:33.670586 16671 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:20:33.670167 23040 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.674633 16670 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:20:33.643415 23111 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:33.654922 22692 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:20:33.654664 22685 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:33.686579 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:33.637660 22793 raft_consensus.cc:476] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Not starting pre-election -- already a leader
I20260709 13:20:33.687837 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.688700 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.653967 22786 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:33.693711 22707 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.692101 17419 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:33.693182 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:33.695243 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.694255 22707 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.666950 22926 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:20:33.711251 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 74193.4us
I20260709 13:20:33.698212 22786 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.725176 22786 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:33.725620 23112 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.726235 23112 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.715281 23109 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:20:33.728957 23112 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.695147 22868 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.699575 22793 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:33.780809 22707 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.782455 16664 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:33.720329 22947 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 13:20:33.792572 16668 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:33.705706 22685 raft_consensus.cc:697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:33.796981 22537 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:33.713982 22889 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:20:33.797844 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 83171.2us
I20260709 13:20:33.798770 22962 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:33.799425 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88766.2us
I20260709 13:20:33.799584 22537 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:33.820451 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.820863 22707 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.821512 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.826189 16710 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:33.828073 22707 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.767112 22926 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:33.795258 22793 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:33.836891 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 285487us
I20260709 13:20:33.720732 23111 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.856514 22537 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.833877 23034 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:33.867942 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68692us
I20260709 13:20:33.867631 22707 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:33.869658 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:33.870458 22926 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.873148 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.873667 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.873881 16739 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:33.846422 22793 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:33.877882 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:33.845557 22962 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.844177 22980 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:20:33.852866 22853 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.881687 22853 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.882086 22962 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.834018 22868 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.887426 22537 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:33.890025 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:33.834108 23017 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:20:33.883814 22853 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:33.875399 23034 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:33.894523 23110 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:33.894915 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:33.895290 23111 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.896489 22929 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:20:33.897220 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46281.7us
I20260709 13:20:33.898473 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:33.887029 22864 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:33.819213 22889 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:33.910585 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 193851us
I20260709 13:20:33.914114 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.901326 23034 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:33.915578 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:33.917153 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.914113 22929 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:33.917989 22864 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.918939 22929 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.920915 22864 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:33.921861 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.923996 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:33.925052 23125 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.925596 23125 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.915148 22889 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:33.889931 22685 consensus_queue.cc:237] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.928104 22537 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.928634 22537 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.931816 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:33.933298 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:33.934121 23110 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:20:33.935005 23110 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:33.915400 22873 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:33.939960 22873 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:33.942365 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.913324 23109 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:33.943594 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:33.913280 23111 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:33.934764 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:33.952263 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.964394 16590 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:33.930706 22537 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:33.938298 22889 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:33.988664 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:33.989343 17409 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:20:33.981252 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 47753.1us
I20260709 13:20:33.980095 22869 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:33.982579 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:33.984050 17413 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:33.994582 22910 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:33.994616 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:33.998185 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 84858.7us
I20260709 13:20:33.986778 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:33.999687 17416 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:34.032063 17419 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:33.995702 22692 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:34.036413 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.038025 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.045382 16591 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
W20260709 13:20:34.047423 23116 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.172s	user 0.007s	sys 0.010s
W20260709 13:20:34.048925 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.049717 16634 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.074504 17374 leader_election.cc:400] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:20:33.995489 23109 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.036222 23040 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:33.995383 22786 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.084134 16594 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:34.093271 17409 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:34.095605 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:34.096333 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 161696us
I20260709 13:20:34.094414 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.114216 16590 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:34.100911 17374 leader_election.cc:403] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:34.104444 22692 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:34.127437 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 77798.4us
I20260709 13:20:34.136125 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:20:34.139500 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 195279us
I20260709 13:20:34.124161 22869 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:34.119611 22868 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.125622 23004 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:20:34.140388 16634 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:34.146113 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109206us
I20260709 13:20:34.113127 23040 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.140722 23127 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:34.129115 22943 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:20:34.150946 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:34.160288 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.161403 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:20:34.168777 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44282.5us
I20260709 13:20:34.150524 23127 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.141191 23034 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.172060 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:34.193660 16594 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:34.173318 23127 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:34.163740 23112 raft_consensus.cc:2764] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:20:34.169975 22889 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.201508 23143 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:34.199227 22869 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.208081 23004 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15.
I20260709 13:20:34.208391 23127 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:34.210579 23132 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:20:34.222282 23133 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:20:34.211362 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:34.213878 23131 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:20:34.214267 23128 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:34.223129 23120 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:20:34.223176 16596 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.223546 23128 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.224025 23128 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.224186 23141 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:20:34.226809 23040 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.226994 23109 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.218250 23143 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.228248 23143 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:34.214478 16670 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
W20260709 13:20:34.214995 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 64144.5us
I20260709 13:20:34.219798 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:34.231156 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:34.240252 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85231.5us
I20260709 13:20:34.241201 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:34.243115 23118 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:34.243896 23118 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:34.246325 22786 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:34.247659 23130 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 13:20:34.248027 23034 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.248929 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.249676 16589 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
W20260709 13:20:34.249739 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 50821.7us
I20260709 13:20:34.251238 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.251984 23109 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:34.252487 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.262624 23110 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:34.200126 22692 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.265236 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:34.257252 23125 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.266176 23110 raft_consensus.cc:2764] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:20:34.273383 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:34.279055 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:34.279728 17413 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:34.280687 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:34.281334 16744 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:34.281962 22777 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:34.283150 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:34.283980 23034 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:34.284554 23110 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:34.285619 23109 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:34.286069 23109 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:34.286556 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:34.292548 23110 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:34.258989 23040 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:34.293726 23040 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.294134 23040 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:34.300690 22685 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:34.301524 23117 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.424s	user 0.003s	sys 0.000s
I20260709 13:20:34.301453 22685 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.302449 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.303092 16667 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:34.257016 22793 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.309301 22793 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.309672 22685 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.293920 23110 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:34.311143 23110 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.311491 22869 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:34.286931 23109 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.314042 23109 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:20:34.313980 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.315626 23112 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:34.316151 23112 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:34.316998 23112 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.317721 22910 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:34.320300 23127 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:34.320991 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 82754.4us
I20260709 13:20:34.286247 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:34.322955 23118 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:34.323012 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.306671 16596 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:34.261260 23128 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.327332 17413 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:34.323973 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.334031 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.340622 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.341382 16671 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:20:34.335811 16710 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:34.346879 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.347527 16741 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:34.348575 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.349421 17409 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:34.349771 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.350246 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:34.350838 17419 raft_consensus.cc:2376] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19. Re-sending same reply.
I20260709 13:20:34.351701 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:34.352491 22957 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:20:34.352620 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.354197 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.354770 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:20:34.354517 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:34.355376 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:34.355984 16740 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.355931 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.357697 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.358565 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:34.359109 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:34.359213 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.359541 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.354507 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.360544 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:34.352589 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.361334 16742 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:34.359386 22926 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:34.355523 16594 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.323460 23118 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:34.354856 16593 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:34.359124 16669 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:34.359758 16595 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:34.362300 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:34.344141 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.368989 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:34.364460 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:34.365126 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.370118 17413 raft_consensus.cc:2376] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 9. Re-sending same reply.
I20260709 13:20:34.361428 16737 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:34.370666 23118 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.372239 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.374202 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:34.373240 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 87069.8us
I20260709 13:20:34.362144 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:34.368588 16487 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 15 to 17, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:34.375414 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.379400 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.379565 16745 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:34.341382 22910 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:34.370394 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 312) took 1788 ms. Trace:
I20260709 13:20:34.380378 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.380908 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.381733 22537 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:34.387483 17411 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:34.389178 17409 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:34.381903 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:34.391711 16741 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 11: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:34.383322 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.383322 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.384635 17413 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:34.372084 22926 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:34.403838 23017 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:34.403549 22926 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.404410 23017 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.406395 23017 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.387444 22537 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.407929 23128 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.409096 22537 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.318675 23143 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.340356 23127 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:20:34.367250 23111 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:34.378026 16487 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 14, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:34.417361 16595 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:34.415093 16740 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
W20260709 13:20:34.413811 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 69746.7us
W20260709 13:20:34.419998 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58285.7us
W20260709 13:20:34.420343 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 46232.1us
I20260709 13:20:34.423053 16669 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:20:34.418252 16487 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 16, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 16 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:34.420791 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.411550 22889 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:34.431128 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.434965 16634 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.435122 16664 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 23.
I20260709 13:20:34.436101 16634 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.386116 22910 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.426730 16487 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 12 to 14, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:34.414422 22957 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:34.443207 22957 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:34.443688 22957 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:34.445822 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.413825 23142 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:20:34.447054 22868 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:34.448012 23142 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:34.433878 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.447692 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.436483 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.451099 16742 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:34.439697 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.420024 22853 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:34.455057 22853 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.455490 22853 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:34.447674 23040 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.462191 23040 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.464018 22869 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:34.464560 22869 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.465385 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.466303 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.467306 22685 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:34.467770 22685 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:34.468690 22685 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.469506 22864 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:34.470000 22864 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.470446 22864 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:34.471045 23135 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:34.471468 23135 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.471786 23135 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:34.473901 16634 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.475035 22869 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.475406 22868 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:34.442050 23113 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:34.475991 22868 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:34.476399 23113 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.477151 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.476930 22868 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.477689 17413 raft_consensus.cc:1813] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Handling vote request from an unknown peer efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.478163 17413 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:34.479126 16668 raft_consensus.cc:2368] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16.
I20260709 13:20:34.480264 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:34.480839 16664 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:34.453858 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:34.481714 16740 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.482319 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.483021 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.487072 22889 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:34.480747 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:34.386509 16489 rpcz_store.cc:276] 0709 13:20:32.557392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.557654 (+   262us) service_pool.cc:224] Handling call
0709 13:20:34.345240 (+1787586us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:33.101258 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:33.102001 (+   743us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:33.102058 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:20:33.102080 (+    22us) tablet.cc:662] Decoding operations
0709 13:20:33.102719 (+   639us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:33.102807 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:33.102840 (+    33us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:20:33.103486 (+   646us) tablet.cc:701] Row locks acquired
0709 13:20:33.103545 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:20:33.103721 (+   176us) write_op.cc:270] Start()
0709 13:20:33.103865 (+   144us) write_op.cc:276] Timestamp: P: 1783603233103666 usec, L: 0
0709 13:20:33.103904 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:20:33.104575 (+   671us) log.cc:844] Serialized 1284 byte log entry
0709 13:20:33.800574 (+695999us) op_driver.cc:464] REPLICATION: finished
0709 13:20:33.883926 (+ 83352us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327cada0ac 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f328361aef0
0709 13:20:34.218706 (+334780us) write_op.cc:301] APPLY: starting
0709 13:20:34.227461 (+  8755us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:34.239778 (+ 12317us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:34.239816 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:34.241841 (+  2025us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:34.242204 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:20:34.242251 (+    47us) write_op.cc:312] APPLY: finished
0709 13:20:34.242555 (+   304us) log.cc:844] Serialized 41 byte log entry
0709 13:20:34.242794 (+   239us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:34.268648 (+ 25854us) write_op.cc:454] Released schema lock
0709 13:20:34.268710 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:34.269174 (+   464us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:34.269200 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":362368,"apply.run_cpu_time_us":6985,"apply.run_wall_time_us":70501,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16679,"num_ops":3,"prepare.queue_time_us":209186,"prepare.run_cpu_time_us":18410,"prepare.run_wall_time_us":185093,"raft.queue_time_us":340448,"raft.run_cpu_time_us":17768,"raft.run_wall_time_us":340780,"replication_time_us":696484,"spinlock_wait_cycles":26813184,"thread_start_us":920070,"threads_started":4,"wal-append.queue_time_us":352765}]]}
I20260709 13:20:34.490265 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.449404 23128 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:34.490520 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.490756 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:34.488448 16667 raft_consensus.cc:2368] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:20:34.491488 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:34.489241 16737 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:20:34.506762 16669 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:34.500710 16740 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:34.514429 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 66799.4us
I20260709 13:20:34.516400 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:34.521922 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:34.547905 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:34.549990 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:34.552672 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:34.553555 17411 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:34.556138 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.563577 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:34.442902 22910 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:34.578454 16742 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:34.578743 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:34.579018 17409 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:34.588791 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.591900 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.599511 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:34.545506 23128 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.558772 23136 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:34.607888 22707 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:34.627920 16664 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:34.628885 16667 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:34.616398 17411 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:34.564458 22692 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:34.580361 22929 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:34.641448 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:34.649364 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.647693 16737 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:34.647321 16669 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:34.655205 23112 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:34.655975 23112 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:34.657140 23112 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:34.659521 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 238814us
I20260709 13:20:34.660630 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.465412 23118 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.624176 23128 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:34.578122 23017 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:20:34.642781 22929 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:34.641072 22873 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:34.649314 23141 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:34.686331 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 195649us
I20260709 13:20:34.693308 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:34.693846 16662 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:34.702342 16737 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:34.671783 23113 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:34.677644 23142 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:34.707054 23153 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:34.718210 16594 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:34.703961 16663 raft_consensus.cc:2376] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18. Re-sending same reply.
I20260709 13:20:34.700838 22929 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:20:34.748586 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 200753us
I20260709 13:20:34.766731 23118 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.774222 22864 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:34.775478 23148 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:20:34.775837 23143 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:34.776002 22793 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:34.722302 23136 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:20:34.776564 22873 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index 9 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:34.780107 16587 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:20:34.789314 16708 leader_election.cc:400] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:20:34.849782 16708 leader_election.cc:403] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:34.707635 23153 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.703852 22889 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.860829 16665 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:20:34.861907 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 72817.3us
I20260709 13:20:34.872148 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:20:34.871184 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 227475us
I20260709 13:20:34.879662 16594 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:20:34.886294 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 225774us
I20260709 13:20:34.887209 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.894464 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:34.897405 17374 leader_election.cc:400] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16.
I20260709 13:20:34.903975 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:34.865837 23118 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:20:34.811777 23135 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:34.954813 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:34.868906 22853 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:34.992398 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 93736.9us
I20260709 13:20:34.917150 22869 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:34.994661 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:34.920727 23109 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:34.999343 17411 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:35.001243 17409 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:34.951884 22692 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.014181 17411 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:35.015981 16671 raft_consensus.cc:2368] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16.
I20260709 13:20:35.016215 16670 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:35.030246 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.032271 16636 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:35.033604 22868 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:35.035140 22868 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:35.037742 22868 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:35.038292 22868 raft_consensus.cc:697] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:35.040233 22868 consensus_queue.cc:237] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:34.997138 23142 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:35.055830 23150 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 13:20:35.057528 23151 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:20:35.057704 23160 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:35.058318 23113 raft_consensus.cc:2764] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:20:35.058560 23160 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.059520 23113 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.059638 23185 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:35.060549 23154 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:20:35.062394 16590 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:35.060145 23113 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.065491 23113 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.066155 23113 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:35.060200 23185 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.067121 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.067983 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 23.
I20260709 13:20:35.069152 23185 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.070538 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.071753 16590 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:35.074657 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.077064 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.077728 16591 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:35.078761 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:35.079385 16592 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:35.086757 16663 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:20:35.094836 16592 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:35.102927 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.108657 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:35.052673 23175 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:35.086241 23185 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.024524 22692 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.066781 23113 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.096261 23142 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:35.170468 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 168381us
W20260709 13:20:35.170578 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 138375us
I20260709 13:20:35.144202 22864 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.171418 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:35.177568 22692 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.143208 23185 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.180892 23185 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.184047 23142 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:35.187343 22957 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.743s	user 0.010s	sys 0.000s
I20260709 13:20:35.187744 23153 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.187815 22957 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.189064 17412 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:35.203549 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:20:35.203559 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 94998.5us
I20260709 13:20:35.206182 17419 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:20:35.207782 17374 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.211644 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.213871 23176 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:35.213977 23169 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.152s
I20260709 13:20:35.214612 23160 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.215766 23169 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:20:35.216326 23169 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:20:35.218578 23173 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 13:20:35.218746 22910 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.000s
W20260709 13:20:35.221851 23128 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.517s	user 0.007s	sys 0.000s
I20260709 13:20:35.226982 23149 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:35.252168 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:35.265355 17419 raft_consensus.cc:1275] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:35.281299 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:35.266420 17409 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:35.214696 23176 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:35.261415 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.265707 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:35.282786 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:35.287039 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.287604 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:35.287873 16591 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:35.285467 17415 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:35.291982 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:35.301342 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 81876.7us
I20260709 13:20:35.233155 23161 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:20:35.227946 22929 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.217258 23169 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.309777 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:35.252985 23199 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:35.277776 23153 consensus_queue.cc:1059] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 13:20:35.319882 23170 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:35.320792 23178 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.321334 23178 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.323562 23178 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.325265 22929 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.325551 23161 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:35.253046 22910 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.278035 23128 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.334928 16590 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:35.323562 23113 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.332325 22929 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:35.331931 23170 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:35.353040 23170 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:35.354089 23182 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:20:35.341804 22889 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:35.373952 16589 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:35.374799 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:35.375211 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:35.375524 16595 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:35.375902 16589 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:35.379196 23188 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:20:35.382932 22889 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:35.384078 22889 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.385838 16589 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:35.387166 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.387962 16589 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:35.391577 23187 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:20:35.350419 22929 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:35.409525 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:35.347029 23127 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:20:35.394116 22957 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.419378 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:35.429080 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:35.431334 16595 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
W20260709 13:20:35.433194 23118 consensus_meta.cc:287] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.000s
I20260709 13:20:35.433025 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:35.433619 23118 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14.
I20260709 13:20:35.433779 17420 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:35.455483 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:35.454612 23179 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:35.456724 23189 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:35.457149 23189 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.457485 23189 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.457830 22957 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:35.458252 22957 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.458670 22957 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:35.437655 22947 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.459290 22947 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.461694 22947 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.462846 16591 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:35.468940 23118 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.469650 23118 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.437848 23143 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:20:35.396909 23176 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:35.474153 23165 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.417s	user 0.002s	sys 0.001s
I20260709 13:20:35.475208 23118 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.475400 23153 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:35.474478 23128 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.477551 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:35.418534 23181 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:35.456748 23179 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.483286 23181 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:35.483547 23179 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:35.483616 17414 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:20:35.482719 22869 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:35.491101 23194 consensus_queue.cc:1048] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:35.492130 22869 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:35.492624 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 1001) took 1141 ms. Trace:
I20260709 13:20:35.493022 17416 rpcz_store.cc:276] 0709 13:20:34.351361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.352810 (+  1449us) service_pool.cc:224] Handling call
0709 13:20:34.353533 (+   723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:34.353865 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:34.353888 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:34.491936 (+138048us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b2c0004cd68. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:20:34.565884 (+ 73948us) log.cc:844] Serialized 548 byte log entry
0709 13:20:34.750853 (+184969us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:34.750906 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:34.750992 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:35.473374 (+722382us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:35.492362 (+ 18988us) raft_consensus.cc:1685] finished
0709 13:20:35.492385 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:35.492526 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28461,"spinlock_wait_cycles":77862144,"thread_start_us":151024,"threads_started":1,"wal-append.queue_time_us":490580}
I20260709 13:20:35.482990 22929 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.497475 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1849) took 1047 ms. Trace:
I20260709 13:20:35.497850 17412 rpcz_store.cc:276] 0709 13:20:34.450270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.450434 (+   164us) service_pool.cc:224] Handling call
0709 13:20:34.475281 (+ 24847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:35.189449 (+714168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.497245 (+307796us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54006c02d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:35.497416 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641134464}
I20260709 13:20:35.499660 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.507303 22853 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:35.510545 23193 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.454s	user 0.002s	sys 0.000s
I20260709 13:20:35.511499 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:35.512722 17414 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:35.510759 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:35.515317 22943 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:35.519117 16670 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:35.525705 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.526374 16740 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:20:35.512936 23135 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.527412 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:35.539943 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.540751 16745 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:35.540066 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:35.551501 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 246543us
I20260709 13:20:35.548588 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.560846 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.561565 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.560956 16740 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 9 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:35.582491 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:35.580313 16708 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:35.583932 16737 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:35.586530 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.586580 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.523705 23153 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.557845 22980 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:35.576211 23196 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:35.577683 23128 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:35.563828 23021 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:35.602408 22980 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:35.609586 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:35.544636 22926 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:20:35.612669 17414 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:35.617849 16670 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
W20260709 13:20:35.624388 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84361.6us
I20260709 13:20:35.629724 16744 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:35.631265 16739 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:35.631634 23203 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:35.511485 23181 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.599720 23136 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:35.576767 23184 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 13:20:35.604171 23196 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:35.639168 23136 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:35.651072 23159 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:35.650941 23136 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.632170 23203 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.657051 23203 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.673023 23211 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:35.673516 23211 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:35.674011 22980 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.633370 23021 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.635433 16736 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "73089175ee9c465197a1a11e304346be"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:46132
I20260709 13:20:35.676236 16736 raft_consensus.cc:606] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:35.677402 23021 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.638548 23196 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:35.687625 23159 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:35.689050 23159 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.635782 23199 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:20:35.702076 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:35.702911 16736 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:35.638161 22957 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.669962 23034 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:35.711066 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:35.710364 16744 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:35.711795 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 1252) took 1346 ms. Trace:
I20260709 13:20:35.725587 16587 rpcz_store.cc:276] 0709 13:20:34.358948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.369689 (+ 10741us) service_pool.cc:224] Handling call
0709 13:20:35.705407 (+1335718us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54000514d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:35.705531 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":890853760}
I20260709 13:20:35.674296 23211 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.700014 23179 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.741104 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:35.745592 23208 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:20:35.747290 23179 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.748723 22910 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.751209 16670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:35.751832 16670 raft_consensus.cc:606] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:35.752166 16670 raft_consensus.cc:614] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:35.754485 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 3575) took 1272 ms. Trace:
I20260709 13:20:35.754964 23211 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:35.756103 16665 rpcz_store.cc:276] 0709 13:20:34.481974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.489911 (+  7937us) service_pool.cc:224] Handling call
0709 13:20:35.754279 (+1264368us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540033ac54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:35.754412 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823348608}
I20260709 13:20:35.748857 22957 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:35.760010 23178 raft_consensus.cc:3055] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:35.760532 23178 raft_consensus.cc:740] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:35.761432 23178 consensus_queue.cc:260] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.16, Last appended by leader: 16, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.763643 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:35.764237 16738 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:35.766562 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:35.726198 23034 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.767876 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:20:35.769168 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:35.769562 23034 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.769711 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:35.769874 16595 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 24: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:35.773591 16738 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:35.778309 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:35.779644 23166 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:35.780192 23166 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:35.781100 23166 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.782688 16736 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
W20260709 13:20:35.785086 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.787361 16670 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:20:35.790237 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:35.791527 16670 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 24: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:35.795071 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:35.795894 16669 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:35.699478 23189 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.816110 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:35.817104 16742 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:35.818511 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:35.785271 23153 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.823524 16662 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:20:35.839697 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:35.840608 16736 raft_consensus.cc:2393] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 24: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:35.842494 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:35.840286 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.813273 23125 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:35.844725 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.848637 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:35.849773 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3577) took 1363 ms. Trace:
I20260709 13:20:35.851436 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:35.852440 16664 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 15: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:35.850221 16667 rpcz_store.cc:276] 0709 13:20:34.482565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.587588 (+105023us) service_pool.cc:224] Handling call
0709 13:20:34.628576 (+ 40988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:34.629217 (+   641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:35.845744 (+1216527us) spinlock_profiling.cc:243] Waited 8.76 ms on lock 0x7b54001f6b50. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:35.845873 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20156800}
I20260709 13:20:35.855085 22537 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:35.855479 22537 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:35.833578 23211 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:35.856127 22537 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.857082 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.862201 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.860366 23170 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:35.852778 23158 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:35.866820 23176 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.861222 23110 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:20:35.867431 23176 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.851966 23127 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:20:35.869431 23176 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.857406 23125 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.870626 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:35.871554 23125 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.871799 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:35.853120 23118 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:35.873060 23118 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:35.873512 16662 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:20:35.875046 23118 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.875110 16662 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 9 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:35.878433 23211 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:35.880769 23211 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:35.881457 23122 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:35.882068 23122 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:35.882640 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:35.886561 16559 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:35.891484 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:35.878607 23181 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:35.896217 16742 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:35.889564 23110 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:35.893548 23112 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:35.897545 23216 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:20:35.899273 23111 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:35.900980 22873 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:35.901731 23169 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:20:35.906682 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.884418 23153 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:35.907644 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:35.915383 22962 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:35.916007 22962 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:35.916922 22962 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:35.922571 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:35.922683 23226 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:20:35.923960 23218 log.cc:927] Time spent T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:20:35.926820 16742 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:35.928886 22537 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:35.962041 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:35.961169 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:35.968184 16667 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 15: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:35.963090 23189 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:35.911288 23122 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:35.994297 16671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 3567) took 1555 ms (client timeout 1678 ms). Trace:
W20260709 13:20:35.994803 16671 rpcz_store.cc:269] 0709 13:20:34.439181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:34.439408 (+   227us) service_pool.cc:224] Handling call
0709 13:20:35.994080 (+1554672us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b540033b654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:35.994226 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1290399616}
W20260709 13:20:35.999504 23235 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:20:36.002586 23196 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.005010 23196 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:36.007243 23234 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:20:36.016417 23135 raft_consensus.cc:1064] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:20:36.019299 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:36.019948 16592 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:36.026585 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:36.034065 16592 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:36.047174 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:36.047788 17410 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.048399 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:36.048899 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:36.049006 17411 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:36.045765 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:36.055325 17409 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:36.039170 23122 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:36.054998 23135 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.041801 23128 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:36.059929 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.061935 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:36.065713 22929 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:36.066469 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:36.070057 23162 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:36.070910 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.055414 22782 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:36.055002 23169 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:36.073405 16671 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:36.073761 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.076459 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.081485 16596 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:36.078428 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.080403 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.081629 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.070982 23166 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:36.085238 22707 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:20:36.092950 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.100631 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:36.101943 22926 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:36.102447 22926 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:36.108229 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.106611 23171 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:36.119314 23171 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:36.119810 23171 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:36.109226 22926 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:36.127456 22853 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:36.084462 23236 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:36.087989 23128 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:36.099431 23214 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:36.136914 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.081640 23169 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.105226 23166 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:36.142304 23128 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.143260 23214 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.085825 22782 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.149611 16740 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:20:36.153709 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.162655 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:36.163668 23246 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:20:36.163913 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:36.164544 17419 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:36.165488 17419 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: []
I20260709 13:20:36.167831 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:36.168469 23220 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.428s	user 0.003s	sys 0.000s
I20260709 13:20:36.168680 16745 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:36.158671 23161 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:20:36.172082 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:36.140012 23236 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.157550 23113 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.175159 23162 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:36.175638 23162 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:36.176120 23162 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:36.176389 22853 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:36.176700 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:36.179977 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.180248 23166 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.180344 17419 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:36.181560 17415 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: [16.16-16.16]
I20260709 13:20:36.191385 16743 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 8.
I20260709 13:20:36.161351 17411 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 9 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:36.175287 23113 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.218518 17410 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:36.156733 16708 leader_election.cc:400] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16.
I20260709 13:20:36.223316 16708 leader_election.cc:403] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:36.223501 16744 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:20:36.226176 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 69582.6us
I20260709 13:20:36.227838 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:36.233341 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:36.270982 22707 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:36.273881 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:36.176185 23214 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:36.248466 23213 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16.
W20260709 13:20:36.277475 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51850.3us
W20260709 13:20:36.277150 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49354us
I20260709 13:20:36.174803 23236 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.282615 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:36.272667 23153 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:36.296633 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:36.309557 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:36.311604 22685 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:36.312120 22685 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.314117 22685 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:36.314550 22685 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:36.315346 22782 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.317297 23110 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:36.319973 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:36.321237 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:36.290150 23111 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:36.325752 22707 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:36.326370 22707 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:36.326686 22707 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.328567 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.329371 16594 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 8.
I20260709 13:20:36.331976 22685 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:36.333284 23161 raft_consensus.cc:993] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:36.333789 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.337054 23161 raft_consensus.cc:1081] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:36.337285 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.332679 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:36.344225 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.297626 22873 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.321614 23236 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:20:36.325322 23191 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.328150 23111 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:36.277781 23160 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:20:36.335184 23213 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:36.342701 22685 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.336017 23169 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:36.318027 23110 consensus_queue.cc:1059] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:20:36.360044 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:36.349334 23227 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:20:36.354027 23111 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:36.368165 23249 log.cc:927] Time spent T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:20:36.350188 23113 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:36.358911 16588 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "73089175ee9c465197a1a11e304346be"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38582
I20260709 13:20:36.397114 16588 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:36.397485 16588 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.398999 23153 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:36.399554 23153 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.401700 23153 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:36.403190 23004 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:36.403745 23004 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.405802 23004 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.405979 23171 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.381803 23158 consensus_queue.cc:1059] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:20:36.351342 22873 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.379287 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 3281) took 1306 ms. Trace:
I20260709 13:20:36.411523 16590 rpcz_store.cc:276] 0709 13:20:35.072964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:35.074297 (+  1333us) service_pool.cc:224] Handling call
0709 13:20:36.379036 (+1304739us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400157a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:36.379221 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595777920}
I20260709 13:20:36.414968 23189 raft_consensus.cc:3055] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:20:36.415454 23189 raft_consensus.cc:740] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:36.416306 23189 consensus_queue.cc:260] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.354763 23191 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.408460 16492 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 10 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 19 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:36.395350 23110 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:36.418728 23110 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:36.419437 23191 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:36.419622 23110 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:36.422093 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:36.418310 23241 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.447s	user 0.001s	sys 0.000s
I20260709 13:20:36.393518 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:36.441594 16594 raft_consensus.cc:2393] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:36.442804 16487 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 305) took 3599 ms. Trace:
I20260709 13:20:36.431640 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:36.448742 16588 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.452598 22707 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:36.454733 22707 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.446316 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:36.466390 16596 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:36.468166 16492 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:36.471325 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:36.473460 23172 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:36.474387 23172 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:36.479715 16588 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:36.382614 23227 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:36.470755 16492 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 17, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:36.489588 16492 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:36.492411 16492 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 13, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:36.443434 16487 rpcz_store.cc:276] 0709 13:20:32.787507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:32.834972 (+ 47465us) service_pool.cc:224] Handling call
0709 13:20:36.386811 (+3551839us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:34.917671 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:35.153474 (+235803us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:35.159666 (+  6192us) write_op.cc:435] Acquired schema lock
0709 13:20:35.165147 (+  5481us) tablet.cc:662] Decoding operations
0709 13:20:35.228723 (+ 63576us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:35.228779 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:35.228807 (+    28us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:20:35.229236 (+   429us) tablet.cc:701] Row locks acquired
0709 13:20:35.229250 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:20:35.229394 (+   144us) write_op.cc:270] Start()
0709 13:20:35.229501 (+   107us) write_op.cc:276] Timestamp: P: 1783603235229346 usec, L: 0
0709 13:20:35.229522 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:20:35.230279 (+   757us) log.cc:844] Serialized 1724 byte log entry
0709 13:20:35.527736 (+297457us) op_driver.cc:464] REPLICATION: finished
0709 13:20:35.695954 (+168218us) write_op.cc:301] APPLY: starting
0709 13:20:35.702177 (+  6223us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:35.750728 (+ 48551us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:35.788509 (+ 37781us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:35.876818 (+ 88309us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:35.877036 (+   218us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:20:35.877095 (+    59us) write_op.cc:312] APPLY: finished
0709 13:20:35.877319 (+   224us) log.cc:844] Serialized 49 byte log entry
0709 13:20:35.877498 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:35.877600 (+   102us) write_op.cc:454] Released schema lock
0709 13:20:35.877637 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:35.878056 (+   419us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:35.878083 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":137776,"apply.run_cpu_time_us":8815,"apply.run_wall_time_us":196647,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20863,"num_ops":4,"prepare.queue_time_us":324990,"prepare.run_cpu_time_us":12823,"prepare.run_wall_time_us":328698,"replication_time_us":249843,"thread_start_us":288927,"threads_started":2}]]}
I20260709 13:20:36.496954 23227 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:36.579895 23111 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.591213 22873 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:36.605093 22929 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.006s
I20260709 13:20:36.482246 22926 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.609669 22926 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.611169 16736 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:20:36.556604 23192 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.615934 23192 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.618477 23192 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.621253 16591 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:36.626951 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:36.632226 22707 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:36.640079 16744 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: []
W20260709 13:20:36.640767 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:36.649673 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.642385 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:36.650323 16743 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:20:36.644775 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 335270us
I20260709 13:20:36.640110 16736 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:36.653066 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:36.664407 16742 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:36.652828 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.665822 22707 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:36.667428 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.679745 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.685032 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:36.685807 16669 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.687551 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.689567 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:36.690194 17410 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:36.691781 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:36.692404 17411 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:36.698571 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:36.700402 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.683637 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.701345 16742 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:20:36.703225 16736 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:20:36.703647 16736 pending_rounds.cc:85] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 9
I20260709 13:20:36.703890 16736 pending_rounds.cc:107] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:20:36.704085 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.704126 16736 raft_consensus.cc:2889] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:36.710373 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:36.582736 23111 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:36.724953 17410 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:36.725569 17416 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 14 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:20:36.695883 23162 consensus_meta.cc:287] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 13:20:36.702567 23176 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.745837 17420 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:20:36.752091 22707 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:36.758950 16669 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:36.743327 23162 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.785899 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.787113 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:36.782491 22707 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:36.747056 23111 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:36.789798 22707 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:36.821398 23220 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:20:36.822296 23149 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:36.823009 23149 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.825594 23149 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:36.827008 23239 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:36.827883 23111 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.752151 23176 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.853942 22873 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:36.854696 22873 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:36.862732 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:36.864802 16736 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:36.897598 16669 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:36.914371 17411 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:36.783607 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:36.720865 23180 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
W20260709 13:20:36.940160 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 98932.9us
W20260709 13:20:36.946115 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 244216us
I20260709 13:20:36.947000 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:37.013561 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:37.014040 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:37.026346 23161 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:37.027066 23161 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:37.055476 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 271904us
W20260709 13:20:37.080355 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:36.947608 22910 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:37.161222 23227 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.566s	user 0.004s	sys 0.004s
I20260709 13:20:37.161734 23227 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.162818 16744 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:37.120591 23196 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:37.032277 23111 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.166023 23196 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.120836 23166 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.175031 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:37.205845 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:37.205930 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 259047us
I20260709 13:20:37.121083 23236 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:37.209209 16745 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:37.218433 16745 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:20:37.223412 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 432430us
I20260709 13:20:37.225755 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:37.228662 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:37.229069 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:37.229254 16666 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:37.232817 16736 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
W20260709 13:20:37.238922 16740 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.005s	sys 0.009s
W20260709 13:20:37.241346 16740 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.005s	sys 0.009s
I20260709 13:20:37.247238 23180 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:37.248049 23180 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.251838 23180 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.256671 23285 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:37.257694 23285 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.261193 23285 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.261992 23285 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.262526 23285 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:37.264854 23285 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.268059 23180 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:37.268435 16737 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.012s	sys 0.015s
W20260709 13:20:37.268848 16737 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.012s	sys 0.015s
I20260709 13:20:37.269013 23272 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.268662 23180 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.271656 23180 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:37.278973 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 286835us
I20260709 13:20:37.277300 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:37.281971 22707 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.283224 22707 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:37.283718 22707 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:37.277400 16738 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.000s	sys 0.013s
W20260709 13:20:37.284911 16738 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.001s	sys 0.014s
I20260709 13:20:37.217808 23172 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.234623 23111 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:20:37.305481 16743 consensus_meta.cc:287] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.000s
I20260709 13:20:37.322592 23196 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.323575 23176 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.290762 23161 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.330396 22707 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.287508 23272 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:37.333775 23262 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.502s	user 0.001s	sys 0.010s
I20260709 13:20:37.217621 23166 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.339653 22873 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.345548 16743 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:20:37.174976 22910 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:37.344684 23111 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:37.366837 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 131881us
I20260709 13:20:37.369509 16667 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
W20260709 13:20:37.372208 23112 consensus_queue.cc:1175] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 17 status { last_received { term: 15 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: [16.16-16.16]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:37.372890 23227 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:37.372246 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 143445us
I20260709 13:20:37.352687 23162 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:37.280114 23135 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:20:37.376530 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:37.382556 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 172282us
I20260709 13:20:37.359182 23176 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:37.367431 23111 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:37.349601 23172 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.397939 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:37.352380 23279 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:20:37.398587 22707 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:37.401821 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.402179 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.403383 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.403918 16736 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:37.407900 22707 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:20:37.573244 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 158110us
W20260709 13:20:37.587036 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.395454 23258 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:20:37.596674 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:37.399614 23227 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.604362 16742 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:20:37.573377 23162 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:37.614954 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 358354us
I20260709 13:20:37.616225 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:37.617045 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.593250 23196 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.396653 22962 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.209s
I20260709 13:20:37.398849 23176 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.600857 23125 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:20:37.631881 22910 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.601089 22692 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:20:37.632819 22910 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:37.633369 22910 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:37.637669 23171 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.647890 23166 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:37.616322 23135 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:37.611641 23194 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:20:37.613379 23266 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:20:37.658042 23166 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:37.681483 23166 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.668454 23187 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:37.675722 23272 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.615882 23162 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.613092 23227 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:37.689603 23260 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.420s	user 0.000s	sys 0.001s
I20260709 13:20:37.690016 23290 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:37.690851 23290 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:37.691459 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4193) took 1469 ms. Trace:
I20260709 13:20:37.692000 16739 rpcz_store.cc:276] 0709 13:20:36.222291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.222430 (+   139us) service_pool.cc:224] Handling call
0709 13:20:36.254113 (+ 31683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.270598 (+ 16485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.270642 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.276018 (+  5376us) log.cc:844] Serialized 26 byte log entry
0709 13:20:36.789152 (+513134us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:36.789209 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:36.789311 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.677962 (+888651us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:37.691060 (+ 13098us) raft_consensus.cc:1685] finished
0709 13:20:37.691091 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.691361 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16215,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":88,"spinlock_wait_cycles":67167616,"thread_start_us":486538,"threads_started":1,"wal-append.queue_time_us":968824}
I20260709 13:20:37.693593 23290 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.665894 23279 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:37.623872 23196 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.703003 23286 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:37.754202 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 471419us
W20260709 13:20:37.756913 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.757746 23112 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:37.758442 23112 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:37.761483 23112 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:37.765628 16562 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.008s	sys 0.040s
I20260709 13:20:37.634557 22910 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:37.766238 16562 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.008s	sys 0.040s
I20260709 13:20:37.765754 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4196) took 1106 ms. Trace:
W20260709 13:20:37.770730 23275 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.503s	user 0.008s	sys 0.003s
I20260709 13:20:37.722442 23122 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:37.771616 23122 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:37.772104 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.773316 23149 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:37.773562 23279 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:37.773737 23122 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:37.774821 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.695576 23187 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.773876 23149 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.786494 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:37.787364 16588 raft_consensus.cc:2368] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:37.780215 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:37.781512 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:37.781819 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2625) took 1069 ms. Trace:
W20260709 13:20:37.790000 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 193316us
I20260709 13:20:37.774181 23279 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.777305 23172 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.770654 23180 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:20:37.783453 16594 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:37.783725 23112 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.785246 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:37.786604 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:37.771162 23227 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.897804 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:37.778764 16737 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4197) took 1116 ms. Trace:
W20260709 13:20:37.899072 23295 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:20:37.899484 16592 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:37.750917 23286 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.792781 22910 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:37.771052 16740 rpcz_store.cc:276] 0709 13:20:36.659292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.659439 (+   147us) service_pool.cc:224] Handling call
0709 13:20:36.659717 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:36.659891 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:20:37.242513 (+582622us) log.cc:844] Serialized 14 byte log entry
0709 13:20:37.765396 (+522883us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:37.765447 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:37.765491 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.765561 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.765697 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":579147,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":71306,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2198,"spinlock_wait_cycles":42711808,"thread_start_us":1050653,"threads_started":3,"wal-append.queue_time_us":373352}
I20260709 13:20:37.796800 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:37.868395 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:37.811056 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:37.906523 17410 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
W20260709 13:20:37.853216 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 45606.9us
I20260709 13:20:37.901990 16590 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:37.899411 16737 rpcz_store.cc:276] 0709 13:20:36.662315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.662774 (+   459us) service_pool.cc:224] Handling call
0709 13:20:36.705003 (+ 42229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:36.705215 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:20:37.269249 (+564034us) log.cc:844] Serialized 14 byte log entry
0709 13:20:37.778274 (+509025us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:37.778377 (+   103us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:37.778443 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.778522 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.778672 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59095,"prepare.queue_time_us":534295,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":27,"raft.queue_time_us":86907,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"thread_start_us":1001736,"threads_started":3,"wal-append.queue_time_us":411596}
I20260709 13:20:37.799455 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:37.909657 17413 raft_consensus.cc:2368] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:37.788908 16741 rpcz_store.cc:276] 0709 13:20:36.712355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.712510 (+   155us) service_pool.cc:224] Handling call
0709 13:20:36.719348 (+  6838us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.719643 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.719682 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.720251 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:20:36.867158 (+146907us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:20:37.351496 (+484338us) log.cc:844] Serialized 14 byte log entry
0709 13:20:37.351785 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.351907 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.770114 (+418207us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:37.774527 (+  4413us) raft_consensus.cc:1685] finished
0709 13:20:37.781606 (+  7079us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.781760 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372562,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":29,"raft.queue_time_us":928,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":3315,"spinlock_wait_cycles":45456256,"thread_start_us":532758,"threads_started":2,"wal-append.queue_time_us":544511}
W20260709 13:20:37.853034 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 258129us
I20260709 13:20:37.888073 16591 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:20:37.915869 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 299701us
I20260709 13:20:37.916321 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.885850 23258 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:37.917702 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.919713 16740 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:37.921279 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:37.915073 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:37.924185 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 321677us
I20260709 13:20:37.924925 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:37.925902 16738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.020s	sys 0.021s
W20260709 13:20:37.926367 16738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.020s	sys 0.021s
I20260709 13:20:37.928104 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:37.929009 16669 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:20:37.923542 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:37.897298 23298 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:37.934826 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 165729us
I20260709 13:20:37.814025 23162 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.867100 23160 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:37.801097 23166 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:37.948138 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:37.931053 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4195) took 1272 ms. Trace:
I20260709 13:20:37.941036 16662 raft_consensus.cc:1813] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Handling vote request from an unknown peer efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:37.951493 16662 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:20:37.942214 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:37.902959 23194 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:20:37.811856 23176 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.946699 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.948220 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:37.966440 16587 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:37.948338 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:37.812722 23196 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.915232 23158 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.950173 16738 rpcz_store.cc:276] 0709 13:20:36.658682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.659118 (+   436us) service_pool.cc:224] Handling call
0709 13:20:36.692854 (+ 33736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:36.693102 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:20:37.285193 (+592091us) log.cc:844] Serialized 14 byte log entry
0709 13:20:37.930601 (+645408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:37.930694 (+    93us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:37.930761 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.930889 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:37.930997 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20455,"prepare.queue_time_us":546148,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":3488,"thread_start_us":1190299,"threads_started":2}
I20260709 13:20:37.957545 16737 raft_consensus.cc:1240] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:20:37.961079 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:37.962626 16662 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:37.960961 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:37.812448 23111 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.971617 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.971535 17376 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:37.908725 23112 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.943939 23292 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:37.991019 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:38.007436 16589 raft_consensus.cc:1813] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Handling vote request from an unknown peer efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:38.008289 16589 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:20:38.017890 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:37.977751 23176 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:37.981815 23125 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.069883 23176 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.070345 23176 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:38.070073 23125 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.982266 23122 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.092s
I20260709 13:20:38.072418 23125 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.977463 23287 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:38.072924 23300 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:38.073040 23287 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:37.965716 23166 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.108s
I20260709 13:20:38.074373 23166 consensus_queue.cc:1243] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 17.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:20:38.074895 23166 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:38.075206 23166 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:38.075356 23287 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.971508 23172 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.076263 23172 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:37.974551 23306 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 13:20:38.078364 23172 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.064940 16588 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:38.002074 23307 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:38.066673 16596 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:20:38.064813 23187 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:37.945591 23298 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.002332 23303 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:38.067849 23112 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.972110 23302 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 13:20:38.068404 23149 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:37.954087 23160 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.076119 23166 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.064443 23292 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.064532 23111 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.079106 23307 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.128664 23307 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:38.129148 23298 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:38.095546 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 124081us
I20260709 13:20:38.106967 23166 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:38.111972 23292 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.105011 23171 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.113173 23308 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:38.123306 23187 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:38.125916 23160 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.092183 23294 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.126997 23122 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:38.159533 23122 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.127912 22943 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:37.978160 23158 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.132434 23162 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:38.167544 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 178896us
W20260709 13:20:38.170087 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 248873us
I20260709 13:20:38.173947 16559 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:38.156066 23292 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:38.175513 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 233490us
I20260709 13:20:38.179086 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:38.185487 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:38.147921 23171 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.187522 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:38.180552 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:38.105166 23258 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.193610 16559 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:38.157025 23187 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.205701 16708 leader_election.cc:400] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:38.206094 16708 leader_election.cc:403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:38.191509 23304 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:38.173686 23162 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.189723 23160 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:38.167119 22943 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.211833 23153 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:38.212858 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.192086 23127 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.195748 23303 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:38.195852 23298 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:38.201188 23112 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:38.162662 23294 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:38.241276 16738 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:20:38.243319 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.244438 16671 raft_consensus.cc:1240] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:38.244608 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.207722 23136 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.189519 23292 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.246773 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.247596 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.248051 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.249173 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:38.250456 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:38.235119 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.217799 22926 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:38.252705 23310 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:20:38.220604 23128 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:38.230829 23153 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.226691 23303 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.230082 23127 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.255694 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.330554 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:38.371824 23127 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.329228 16595 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:38.340938 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.221145 23172 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.416106 23172 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.276053 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:38.378913 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.264046 23187 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.264238 23318 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:20:38.444160 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 238576us
I20260709 13:20:38.436679 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.451082 16738 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:38.435729 16665 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:38.255445 23112 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:38.368036 23298 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.365587 23287 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.372473 23303 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:38.483317 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:38.341323 23323 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.368330 23304 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.263480 23136 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.493156 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:38.497807 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:38.412973 23308 raft_consensus.cc:697] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:38.437193 23040 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.437328 23299 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:38.500990 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.502061 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.517156 16738 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:38.491110 23122 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:38.506337 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.508352 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:38.508256 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.512876 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.520390 16666 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 22
I20260709 13:20:38.449496 22962 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.490119 22692 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.515765 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.333055 22926 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.502558 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:38.518024 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 71910us
I20260709 13:20:38.372676 23128 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.500360 23213 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:38.500506 23158 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.491972 23287 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.571103 23172 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.575887 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:38.491196 23298 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.579164 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.513317 23040 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:38.657560 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 159831us
I20260709 13:20:38.619347 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.584932 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.662636 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.615021 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1609) took 1950 ms. Trace:
I20260709 13:20:38.676370 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.502269 23304 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.519796 23294 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.678146 16744 rpcz_store.cc:276] 0709 13:20:36.664676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.664795 (+   119us) service_pool.cc:224] Handling call
0709 13:20:36.665043 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:37.163233 (+498190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:38.584530 (+1421297us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54002788d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:38.614923 (+ 30393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144423808}
I20260709 13:20:38.511502 23323 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.681532 16708 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:38.698621 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 122830us
I20260709 13:20:38.568440 23292 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:38.701928 23333 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:20:38.702013 23325 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:38.703898 23040 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.705382 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.628666 23122 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:38.670733 23317 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.716717 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.718185 16744 raft_consensus.cc:2410] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 25 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:20:38.620325 23299 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:38.726624 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.601783 23213 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.721406 23330 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.677651 23162 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.681492 22943 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.721763 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.723701 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:38.724095 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:38.708825 23166 consensus_meta.cc:287] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.569s	user 0.011s	sys 0.000s
I20260709 13:20:38.727560 16738 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:38.681854 23172 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:38.618681 23327 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.704574 23158 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:38.697296 23329 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.735762 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.744065 16671 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:38.736707 23342 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.748157 23335 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:38.748446 23316 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.324s	user 0.001s	sys 0.000s
I20260709 13:20:38.785861 16591 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:38.736752 23332 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:38.736701 23324 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.330s	user 0.001s	sys 0.000s
I20260709 13:20:38.642633 23319 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.235s
I20260709 13:20:38.584339 22692 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.725834 23122 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.729768 23292 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:38.576717 22962 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.759454 16666 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:20:38.721415 23328 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.762789 23323 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.726322 23299 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:38.736531 23213 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:38.792832 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.785367 23158 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:38.756120 23341 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:38.661676 23308 consensus_queue.cc:237] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.761113 23172 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.784293 23153 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.734565 23325 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.597960 23112 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.745142 23317 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.760771 23330 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.786170 23342 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.856842 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:38.865132 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:38.865810 16590 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:38.826260 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:38.871366 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:38.807193 23331 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:20:38.810773 23136 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.867908 23299 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:38.862614 22692 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.857527 22926 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.806985 23340 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:38.884721 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:38.887239 23158 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.806722 23343 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.850946 23128 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.895807 16590 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:20:38.869695 23304 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.859776 23040 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:20:38.874241 23258 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:38.882591 23337 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:20:38.943912 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:38.945132 16590 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
W20260709 13:20:38.928690 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 44018.2us
W20260709 13:20:38.938742 22910 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.026s	user 0.009s	sys 0.004s
I20260709 13:20:38.786343 23335 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.951717 23345 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.958417 16593 raft_consensus.cc:2410] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:20:38.795048 23329 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.786538 23327 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:38.940030 23273 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.504s	user 0.006s	sys 0.007s
I20260709 13:20:38.907658 23287 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.955483 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:38.943063 23334 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:38.948514 23176 consensus_meta.cc:287] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.878s	user 0.011s	sys 0.002s
I20260709 13:20:38.968009 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:38.956585 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:38.869329 23341 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.946056 23136 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:38.972853 23166 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.973275 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:38.977540 23342 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.973029 23347 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:38.964262 23298 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:38.981889 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:38.992892 23335 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:38.994967 23178 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:38.946535 23258 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:38.880213 23328 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:38.973549 22910 raft_consensus.cc:2764] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:20:39.004269 16745 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.038448 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:38.972119 23176 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:39.039885 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.040601 16590 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:20:38.970928 23345 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:39.043423 23350 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:39.018585 23110 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:38.980742 23323 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:39.048177 17410 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:20:39.048169 17409 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 15. (term mismatch)
I20260709 13:20:39.049026 17409 pending_rounds.cc:85] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 14
I20260709 13:20:39.049345 17409 pending_rounds.cc:107] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 15.15
I20260709 13:20:39.049762 17409 raft_consensus.cc:2889] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:38.948750 23340 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.029973 23122 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.051331 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3596) took 2358 ms. Trace:
I20260709 13:20:39.051756 16670 rpcz_store.cc:276] 0709 13:20:36.686457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.686568 (+   111us) service_pool.cc:224] Handling call
0709 13:20:36.686908 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.702270 (+ 15362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.702308 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.702794 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:20:36.888957 (+186163us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:36.889015 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:36.889122 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:37.816117 (+926995us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b480021c258. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3284b0daa1 00007f3284b04b37 00007f3284b049ca 00007f3284bfc26f 00007f3284bfbf37 00007f3284c1055c 00007f3284c10508 00007f3284c10488 00007f3284c10443 00007f3284c0f6ec 00007f3283b39a96
0709 13:20:38.764208 (+948091us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:39.044567 (+280359us) raft_consensus.cc:1685] finished
0709 13:20:39.044588 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.044728 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15714,"raft.run_cpu_time_us":14429,"raft.run_wall_time_us":1103579,"spinlock_wait_cycles":82286720,"thread_start_us":169623,"threads_started":1,"wal-append.queue_time_us":541108}
I20260709 13:20:39.051553 23323 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:39.055392 23323 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:39.055825 23340 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:38.978701 23136 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.034616 23317 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:39.060770 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.063557 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:39.065245 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.065902 16590 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:38.955637 23343 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.071599 23330 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.072333 23172 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:39.085645 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.072474 16587 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38582 (request call id 3314) took 1138 ms. Trace:
I20260709 13:20:39.060321 23189 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:39.073724 23325 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.075484 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.112941 16741 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:39.113112 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.114104 17419 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:20:39.076586 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:38.980615 23166 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.079663 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.116312 17414 raft_consensus.cc:2368] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 22.
I20260709 13:20:39.086834 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.087078 16587 rpcz_store.cc:276] 0709 13:20:37.937536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.981319 (+ 43783us) service_pool.cc:224] Handling call
0709 13:20:37.981740 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.131474 (+149734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:38.131517 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.131567 (+    50us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:38.131612 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.072147 (+940535us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540010cf54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:39.072200 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.072376 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343805824}
I20260709 13:20:39.090330 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.099108 17413 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:39.100143 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.102212 23287 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:39.039912 22889 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:20:39.155725 17420 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:39.155819 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.166189 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.045101 23335 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:20:39.167027 16587 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:39.173254 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.170266 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.080404 23227 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:39.185300 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:39.175065 23303 consensus_meta.cc:287] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.000s
I20260709 13:20:39.249147 16594 raft_consensus.cc:2410] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 18 because replica has last-logged OpId of term: 16 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 15.
W20260709 13:20:39.200387 23307 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.071s	user 0.010s	sys 0.004s
W20260709 13:20:39.259883 17376 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 2.008s (SENT) [suppressed 2 similar messages]
I20260709 13:20:39.166385 23287 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:39.293852 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.253149 23290 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.279366 23307 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.334365 16742 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:39.333794 23290 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.256830 23303 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.338335 23303 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:39.338783 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.339258 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.339958 16590 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:39.340762 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.340934 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.342142 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.006000 23347 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.346217 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.208377 23227 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:39.347745 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.327337 17413 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:39.346933 16738 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:39.352080 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.352705 17415 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:39.355218 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.356703 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.358139 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.360937 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.364429 17414 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 22: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:39.302047 23258 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.367403 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:39.366503 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 106667us
I20260709 13:20:39.369585 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.376057 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:39.377383 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.378507 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.387012 23122 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:39.393210 23171 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.398164 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.399592 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3179) took 1155 ms. Trace:
I20260709 13:20:39.400206 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.401535 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.402072 16738 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:39.404534 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.363050 23303 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:39.399972 16663 rpcz_store.cc:276] 0709 13:20:38.244571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.250302 (+  5731us) service_pool.cc:224] Handling call
0709 13:20:38.329324 (+ 79022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.370669 (+ 41345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:38.370699 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:38.370740 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:38.370776 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.399360 (+1028584us) spinlock_profiling.cc:243] Waited 41.1 ms on lock 0x7b54000f37d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:39.399422 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:39.399539 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":94611200}
I20260709 13:20:39.407524 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.408548 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.414110 16738 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 11.
I20260709 13:20:39.414808 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.409791 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.417224 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.418226 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.418780 17411 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:39.419786 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.420414 17411 raft_consensus.cc:1813] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Handling vote request from an unknown peer efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:39.420495 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52265.9us
I20260709 13:20:39.421805 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.421945 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:39.407239 23348 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 13:20:39.423779 17411 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:39.425546 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.380548 23341 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:39.426321 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.379345 23335 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:39.445919 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.446679 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 24.
I20260709 13:20:39.447585 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:39.448110 16590 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:39.424257 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:39.426259 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:39.450644 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.441995 23290 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:39.443467 23166 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:39.444970 23227 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:39.445215 23354 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:20:39.446305 23335 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.452049 16636 leader_election.cc:400] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 22.
I20260709 13:20:39.442751 23136 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:39.450372 17414 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:39.453989 23171 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
W20260709 13:20:39.469388 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.454507 23004 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:39.514072 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.471953 23347 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:39.472095 16741 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:39.473264 23329 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.447831 23342 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:39.432076 23328 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:39.422434 23303 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.481346 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.482335 23327 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.482808 22873 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:39.499373 23213 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.003s
I20260709 13:20:39.486193 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.487318 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.488027 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.528766 16665 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:39.483013 23178 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:39.498698 16636 leader_election.cc:403] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:39.500152 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.502959 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:40.149938 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 647050us
I20260709 13:20:39.515262 17411 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:20:39.515188 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.514534 23004 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:39.501202 23269 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 2.239s	user 0.011s	sys 0.003s
I20260709 13:20:39.457016 23287 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:39.492885 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:39.528779 16670 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:39.680116 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.906378 16589 raft_consensus.cc:1275] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 16. Preceding OpId from leader: term: 17 index: 16. (term mismatch)
I20260709 13:20:39.586709 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1267) took 1683 ms. Trace:
I20260709 13:20:39.719226 23332 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:39.560240 23329 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:39.567049 23279 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:40.079276 16710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.048s	sys 0.120s
I20260709 13:20:39.567515 23303 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:39.905419 16594 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 1269) took 1993 ms (client timeout 1967 ms). Trace:
I20260709 13:20:39.581844 23194 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:20:40.068838 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:39.634574 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:39.570183 23323 consensus_meta.cc:287] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
W20260709 13:20:40.128652 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 698584us
I20260709 13:20:39.571153 23176 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.691128 16741 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
W20260709 13:20:39.572420 23299 consensus_meta.cc:287] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.634s	user 0.005s	sys 0.006s
I20260709 13:20:40.220799 16741 pending_rounds.cc:85] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 11
I20260709 13:20:40.220799 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.221328 16741 pending_rounds.cc:107] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:20:40.221588 16741 raft_consensus.cc:2889] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:39.676033 23170 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.574649 23345 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:39.543431 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:39.497502 23127 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:39.577495 23111 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:39.457211 23258 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:20:39.497207 23300 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:40.131783 22962 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:39.573911 22873 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.204427 22537 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 22.
I20260709 13:20:40.204842 16589 pending_rounds.cc:85] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 15
W20260709 13:20:39.575145 23110 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.003s
I20260709 13:20:40.204912 16588 rpcz_store.cc:276] 0709 13:20:37.903542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.903667 (+   125us) service_pool.cc:224] Handling call
0709 13:20:37.903943 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.078958 (+175015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:39.586405 (+1507447us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001fb3d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:39.586632 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369831808}
I20260709 13:20:40.213347 16738 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:20:40.213363 16710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.048s	sys 0.120s
W20260709 13:20:40.369329 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 948030us
W20260709 13:20:39.583670 23318 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.079s	user 0.013s	sys 0.000s
I20260709 13:20:40.203243 23004 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:40.214823 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1270) took 2298 ms. Trace:
I20260709 13:20:40.215881 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4629) took 2277 ms. Trace:
I20260709 13:20:40.204217 23178 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.393308 23178 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:40.216887 23329 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:40.218327 23289 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 2.575s	user 0.009s	sys 0.025s
W20260709 13:20:40.218662 16593 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 3305) took 2432 ms (client timeout 2725 ms). Trace:
I20260709 13:20:40.387841 16595 rpcz_store.cc:276] 0709 13:20:37.935598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.117325 (+181727us) service_pool.cc:224] Handling call
0709 13:20:38.117767 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.415791 (+298024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.212390 (+1796599us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b54000104d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:40.212582 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480316160}
I20260709 13:20:40.396040 16588 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4631) took 2459 ms. Trace:
I20260709 13:20:40.396569 16588 rpcz_store.cc:276] 0709 13:20:37.936139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.395146 (+2459007us) service_pool.cc:224] Handling call
0709 13:20:40.395485 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.395659 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.395684 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.395725 (+    41us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:40.395755 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.395821 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.395964 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:39.908536 23344 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:39.907708 16663 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
W20260709 13:20:40.395251 16593 rpcz_store.cc:269] 0709 13:20:37.781093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.781258 (+   165us) service_pool.cc:224] Handling call
0709 13:20:40.208486 (+2427228us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400339ad4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:40.213006 (+  4520us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2688993024}
W20260709 13:20:40.209383 23362 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:20:40.223452 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.224419 23342 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:40.405766 23328 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:40.226569 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1378) took 1181 ms. Trace:
I20260709 13:20:40.199677 23335 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:40.148029 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:40.225481 23299 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.219404 23332 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.203730 23361 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:40.230655 23170 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:39.583320 23213 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.221457 23194 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:40.410485 23171 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:20:40.364835 16589 pending_rounds.cc:107] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 16.16
I20260709 13:20:40.414335 16589 raft_consensus.cc:2889] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:40.395805 23318 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
W20260709 13:20:40.355012 23308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.034s	sys 0.066s
I20260709 13:20:40.388314 16596 raft_consensus.cc:2410] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:20:40.219274 16594 rpcz_store.cc:269] 0709 13:20:37.911656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.913380 (+  1724us) service_pool.cc:224] Handling call
0709 13:20:39.905102 (+1991722us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003b74d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:39.905329 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2981802240}
W20260709 13:20:40.403909 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 179491us
I20260709 13:20:40.427527 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.224931 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 1384) took 1164 ms. Trace:
I20260709 13:20:40.406455 17416 rpcz_store.cc:276] 0709 13:20:39.045261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.045403 (+   142us) service_pool.cc:224] Handling call
0709 13:20:39.045733 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:39.345596 (+299863us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:39.345628 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:39.345670 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:39.345707 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.226369 (+880662us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54005e1d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:40.226406 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.226520 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":689163520}
I20260709 13:20:40.223067 23323 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.428526 17413 rpcz_store.cc:276] 0709 13:20:39.060627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.063199 (+  2572us) service_pool.cc:224] Handling call
0709 13:20:40.224726 (+1161527us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b54008ef8d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:40.224841 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79537408}
W20260709 13:20:40.429639 23308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.035s	sys 0.066s
W20260709 13:20:40.410296 23280 log.cc:927] Time spent T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 3.149s	user 0.015s	sys 0.023s
I20260709 13:20:40.416023 22537 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:40.370522 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:40.357542 23111 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.387748 16591 rpcz_store.cc:276] 0709 13:20:37.915206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:37.916825 (+  1619us) service_pool.cc:224] Handling call
0709 13:20:38.785620 (+868795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:38.793658 (+  8038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.213145 (+1419487us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b54001fbdd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:40.213283 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1997410688}
W20260709 13:20:40.433431 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63107.8us
I20260709 13:20:40.433995 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4635) took 2452 ms. Trace:
I20260709 13:20:40.434365 16591 rpcz_store.cc:276] 0709 13:20:37.981190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.433222 (+2452032us) service_pool.cc:224] Handling call
0709 13:20:40.433524 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.433704 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.433728 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.433767 (+    39us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:40.433798 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.433847 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.433945 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.431763 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:40.434990 16710 leader_election.cc:400] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17.
I20260709 13:20:40.435942 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4636) took 2366 ms. Trace:
I20260709 13:20:40.436290 16591 rpcz_store.cc:276] 0709 13:20:38.069770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.435233 (+2365463us) service_pool.cc:224] Handling call
0709 13:20:40.435507 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.435660 (+   153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.435687 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.435724 (+    37us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:40.435753 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.435803 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.435895 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.438220 16739 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:40.357751 23127 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.439685 16744 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:20:40.430720 22537 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.456498 16664 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3595) took 3771 ms. Trace:
I20260709 13:20:40.456775 22537 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.439044 22929 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:40.432910 23314 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.459769 23314 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.428849 16595 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:20:40.461970 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4633) took 2517 ms. Trace:
I20260709 13:20:40.462333 16595 rpcz_store.cc:276] 0709 13:20:37.944157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.408881 (+2464724us) service_pool.cc:224] Handling call
0709 13:20:40.428624 (+ 19743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.461652 (+ 33028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.461903 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.463174 23213 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:40.464522 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4639) took 2208 ms. Trace:
I20260709 13:20:40.465118 16595 rpcz_store.cc:276] 0709 13:20:38.255911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.463407 (+2207496us) service_pool.cc:224] Handling call
0709 13:20:40.463911 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.464121 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.464157 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.464214 (+    57us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:40.464267 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.464333 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.464460 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.467155 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4640) took 2139 ms. Trace:
I20260709 13:20:40.467649 16595 rpcz_store.cc:276] 0709 13:20:38.327611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.466249 (+2138638us) service_pool.cc:224] Handling call
0709 13:20:40.466609 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.466764 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.466789 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.466827 (+    38us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:40.466856 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.466941 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.467082 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.365666 23300 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.488493 23170 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.490301 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4226) took 1083 ms. Trace:
I20260709 13:20:40.490782 16740 rpcz_store.cc:276] 0709 13:20:39.406807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.573809 (+167002us) service_pool.cc:224] Handling call
0709 13:20:39.574238 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:39.574651 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:39.574690 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:39.575306 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 13:20:39.742780 (+167474us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:40.072863 (+330083us) log.cc:844] Serialized 14 byte log entry
0709 13:20:40.073396 (+   533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.073514 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:40.490051 (+416537us) raft_consensus.cc:1685] finished
0709 13:20:40.490076 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.490232 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":391198,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":3578,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":229,"thread_start_us":301823,"threads_started":2,"wal-append.queue_time_us":554588}
I20260709 13:20:40.494285 23299 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.439174 23308 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:40.498001 22873 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:40.500437 23323 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:40.454334 23328 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:40.501849 23328 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:40.504558 23300 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:40.407102 23344 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.508764 23344 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.509629 23329 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:40.514117 23127 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.515573 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:40.516340 17419 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:40.445384 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:40.517527 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:40.519713 23328 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.457090 16664 rpcz_store.cc:276] 0709 13:20:36.685111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.685221 (+   110us) service_pool.cc:224] Handling call
0709 13:20:36.685548 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:36.685869 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:36.685897 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:36.720510 (+ 34613us) log.cc:844] Serialized 26 byte log entry
0709 13:20:37.217517 (+497007us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:37.217570 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:37.217860 (+   290us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:40.214312 (+2996452us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:40.437292 (+222980us) raft_consensus.cc:1685] finished
0709 13:20:40.437324 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.456387 (+ 19063us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":572,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":19486,"spinlock_wait_cycles":63063808,"thread_start_us":458930,"threads_started":1,"wal-append.queue_time_us":519756}
I20260709 13:20:40.412755 23361 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:40.524421 23361 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:40.472915 17415 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 20: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:40.453823 16591 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:40.539331 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4638) took 2287 ms. Trace:
I20260709 13:20:40.541883 16591 rpcz_store.cc:276] 0709 13:20:38.251598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.453131 (+2201533us) service_pool.cc:224] Handling call
0709 13:20:40.453483 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.535717 (+ 82234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.539255 (+  3538us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.545094 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:40.555930 23330 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:20:40.558346 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4641) took 2188 ms. Trace:
I20260709 13:20:40.565034 16595 rpcz_store.cc:276] 0709 13:20:38.369687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.471788 (+2102101us) service_pool.cc:224] Handling call
0709 13:20:40.472151 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.557925 (+ 85774us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.557961 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.558017 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:40.558056 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.558122 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.558273 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.558938 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4642) took 2103 ms. Trace:
I20260709 13:20:40.568477 23342 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:40.569247 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 1.11727e+06us
I20260709 13:20:40.569164 23342 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.569358 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4229) took 1146 ms. Trace:
I20260709 13:20:40.571615 16741 rpcz_store.cc:276] 0709 13:20:39.423050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.223695 (+800645us) service_pool.cc:224] Handling call
0709 13:20:40.473492 (+249797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.473715 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:40.476785 (+  3070us) log.cc:844] Serialized 14 byte log entry
0709 13:20:40.568990 (+ 92205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.569049 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:40.569104 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.569183 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.569291 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":362,"prepare.queue_time_us":3240,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":1451,"raft.queue_time_us":2569,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":91842,"threads_started":1,"wal-append.queue_time_us":90876}
I20260709 13:20:40.573374 22853 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:40.575102 23342 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:40.575467 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 424506us
I20260709 13:20:40.575785 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:40.579886 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1271) took 1878 ms. Trace:
I20260709 13:20:40.580453 16591 rpcz_store.cc:276] 0709 13:20:38.701797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.578925 (+1877128us) service_pool.cc:224] Handling call
0709 13:20:40.579306 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.579485 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.579519 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.579572 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:40.579615 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.579684 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.579818 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.577893 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:40.582228 16595 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:20:40.582820 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:40.578612 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4632) took 2638 ms. Trace:
I20260709 13:20:40.587105 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.588071 16664 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:40.590572 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1272) took 1887 ms. Trace:
I20260709 13:20:40.591015 16595 rpcz_store.cc:276] 0709 13:20:38.702102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.581608 (+1879506us) service_pool.cc:224] Handling call
0709 13:20:40.581953 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.586764 (+  4811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.589783 (+  3019us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.592654 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4646) took 1601 ms. Trace:
I20260709 13:20:40.567234 16592 rpcz_store.cc:276] 0709 13:20:38.455537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.558000 (+2102463us) service_pool.cc:224] Handling call
0709 13:20:40.558359 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.558534 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.558562 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.558625 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:40.558667 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.558755 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.558884 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.594231 23299 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:40.564770 22947 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.596449 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:40.597280 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 143336us
I20260709 13:20:40.587150 16593 rpcz_store.cc:276] 0709 13:20:37.939844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.399295 (+2459451us) service_pool.cc:224] Handling call
0709 13:20:40.399641 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.402334 (+  2693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.402365 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:40.405734 (+  3369us) log.cc:844] Serialized 26 byte log entry
0709 13:20:40.578195 (+172461us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:40.578254 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.578368 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:40.578414 (+    46us) raft_consensus.cc:1685] finished
0709 13:20:40.578425 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.578555 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2284,"raft.queue_time_us":5622,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"thread_start_us":172018,"threads_started":1,"wal-append.queue_time_us":166952}
I20260709 13:20:40.594009 23318 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:40.612648 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4651) took 1543 ms. Trace:
I20260709 13:20:40.599864 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.601189 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4227) took 1194 ms. Trace:
I20260709 13:20:40.602589 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.604768 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.614454 16669 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:20:40.583906 22853 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:40.594314 23179 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:40.604514 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.510339 23004 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.609516 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.609890 16592 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:20:40.622694 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4647) took 1586 ms. Trace:
I20260709 13:20:40.623133 16592 rpcz_store.cc:276] 0709 13:20:39.036065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.608625 (+1572560us) service_pool.cc:224] Handling call
0709 13:20:40.608942 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.622356 (+ 13414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.622605 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.625087 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4652) took 1555 ms. Trace:
I20260709 13:20:40.625509 16592 rpcz_store.cc:276] 0709 13:20:39.069884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.624008 (+1554124us) service_pool.cc:224] Handling call
0709 13:20:40.624385 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.624641 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.624681 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.624744 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:20:40.624806 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.624878 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.625019 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.614356 16736 rpcz_store.cc:276] 0709 13:20:39.412009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.211703 (+799694us) service_pool.cc:224] Handling call
0709 13:20:40.474121 (+262418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.474319 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:20:40.566136 (+ 91817us) log.cc:844] Serialized 14 byte log entry
0709 13:20:40.600817 (+ 34681us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.600876 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:40.600928 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.600996 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.601127 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2798,"prepare.queue_time_us":6968,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":1729,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":3923,"thread_start_us":119155,"threads_started":2,"wal-append.queue_time_us":45273}
I20260709 13:20:40.611315 16595 rpcz_store.cc:276] 0709 13:20:38.991648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.591802 (+1600154us) service_pool.cc:224] Handling call
0709 13:20:40.592117 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.592313 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.592340 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.592386 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:40.592424 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.592479 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.592601 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.564740 23330 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:40.596367 23367 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:40.613932 16593 rpcz_store.cc:276] 0709 13:20:39.069536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.611649 (+1542113us) service_pool.cc:224] Handling call
0709 13:20:40.612002 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.612222 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.612259 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.612316 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:40.612362 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.612434 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.612578 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.629554 16592 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:20:40.659981 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4653) took 1589 ms. Trace:
I20260709 13:20:40.660418 16592 rpcz_store.cc:276] 0709 13:20:39.070160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.626661 (+1556501us) service_pool.cc:224] Handling call
0709 13:20:40.629117 (+  2456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.659670 (+ 30553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.659916 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.630795 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.635143 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1615) took 1561 ms. Trace:
I20260709 13:20:40.664522 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:40.666567 17409 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:40.688055 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:40.689987 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:40.691018 16738 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:40.723450 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:40.723753 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:40.724805 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:40.662055 16742 rpcz_store.cc:276] 0709 13:20:39.074143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.091286 (+ 17143us) service_pool.cc:224] Handling call
0709 13:20:39.171132 (+ 79846us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:39.334743 (+163611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.634916 (+1300173us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400049a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:40.635085 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374949760}
I20260709 13:20:40.610430 23299 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:40.731902 16738 raft_consensus.cc:1275] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:20:40.753751 16736 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 8. Current term is 9. Ops: []
I20260709 13:20:40.754949 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:40.756402 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4634) took 2794 ms. Trace:
I20260709 13:20:40.756801 16594 rpcz_store.cc:276] 0709 13:20:37.961879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.426942 (+2465063us) service_pool.cc:224] Handling call
0709 13:20:40.430191 (+  3249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.430557 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.430588 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:40.431090 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:20:40.569910 (+138820us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:40.631661 (+ 61751us) log.cc:844] Serialized 14 byte log entry
0709 13:20:40.631987 (+   326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.632054 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:40.632091 (+    37us) raft_consensus.cc:1685] finished
0709 13:20:40.632103 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.756339 (+124236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2940,"raft.queue_time_us":819,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":102435,"thread_start_us":196959,"threads_started":2,"wal-append.queue_time_us":137120}
I20260709 13:20:40.613628 22947 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.761970 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.655498 23367 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:40.727883 23179 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:40.749366 23299 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:40.731129 23342 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:40.627512 23318 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.630322 22853 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.731356 23286 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:40.781052 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 184303us
I20260709 13:20:40.782337 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4644) took 1836 ms. Trace:
I20260709 13:20:40.772745 23367 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:40.785126 16591 rpcz_store.cc:276] 0709 13:20:38.946394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.589172 (+1642778us) service_pool.cc:224] Handling call
0709 13:20:40.589605 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.589864 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:20:40.656091 (+ 66227us) log.cc:844] Serialized 14 byte log entry
0709 13:20:40.781883 (+125792us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.781957 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:40.782026 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.782122 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.782267 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1262,"thread_start_us":186044,"threads_started":2}
I20260709 13:20:40.796355 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.774504 23342 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:40.797061 16662 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:40.836589 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:40.795825 22926 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:40.784101 23300 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:40.802680 22853 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.799504 23342 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:40.800269 22947 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:40.847774 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.781498 23330 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.850194 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.799618 23361 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.852310 22537 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:40.847947 23004 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:40.857458 23171 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:20:40.842604 22926 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.851043 23305 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:20:40.887391 23266 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:20:40.847138 23300 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:40.896770 23300 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:40.850893 23179 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:40.865163 23330 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:20:40.902439 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1632) took 1557 ms. Trace:
I20260709 13:20:40.866304 23368 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:40.903298 16743 rpcz_store.cc:276] 0709 13:20:39.345281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.898762 (+1553481us) service_pool.cc:224] Handling call
0709 13:20:40.901969 (+  3207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.902147 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.902172 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.902210 (+    38us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:20:40.902241 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.902289 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:40.902384 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:40.888938 16595 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 8 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:40.912513 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 223379us
W20260709 13:20:40.925357 23338 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 2.194s	user 0.020s	sys 0.041s
I20260709 13:20:40.937089 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:40.937601 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 87521.6us
I20260709 13:20:40.937744 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.939785 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.938364 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:40.945940 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:40.949362 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.951969 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:40.956434 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:40.958431 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.959944 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.966060 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:40.958537 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1894) took 1525 ms. Trace:
I20260709 13:20:40.973423 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:40.975117 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.947243 23266 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:40.977799 16743 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:40.978214 22685 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:40.978891 22685 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:40.979401 22685 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:40.951517 23318 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:40.981949 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:40.968677 17411 rpcz_store.cc:276] 0709 13:20:39.433352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.453586 (+ 20234us) service_pool.cc:224] Handling call
0709 13:20:39.503015 (+ 49429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.151342 (+648327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:40.958254 (+806912us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b5400483850. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:40.958458 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30133504}
I20260709 13:20:40.985036 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.966168 23388 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:40.966852 23307 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:40.977906 23266 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:41.011000 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:40.990509 23344 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:40.995885 23300 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:41.020710 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.042943 23305 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:41.044780 17411 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:41.011358 23127 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:41.052706 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 70842.7us
I20260709 13:20:40.995239 23318 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:20:41.009446 23266 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:41.054416 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.055236 17411 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:41.034729 23258 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:41.059795 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.010934 23361 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.045603 22926 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.066460 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.073124 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.075975 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.078383 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.079216 17420 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:41.095446 23391 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:20:41.101948 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.007653 23388 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:20:41.111341 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.119127 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.152586 16671 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:20:41.153756 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.149179 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.156807 23171 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.161197 23171 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:41.163530 23171 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:41.124511 23004 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:41.170922 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.156185 22685 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:41.172489 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.172642 23299 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.169461 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.180617 22537 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.181103 22537 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.181529 22537 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.159488 23323 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:41.226317 22685 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:41.168197 23318 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:41.228523 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:41.227234 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:41.179066 23004 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.240835 16741 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:41.214406 23161 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:41.214743 23160 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.219631 23323 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.174953 23170 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:20:41.271795 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4628) took 3336 ms. Trace:
I20260709 13:20:41.271454 23161 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:41.275511 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 94821us
I20260709 13:20:41.287528 22537 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.242887 23004 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.273015 16589 rpcz_store.cc:276] 0709 13:20:37.935313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.078785 (+143472us) service_pool.cc:224] Handling call
0709 13:20:39.905962 (+1827177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.414836 (+508874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.271565 (+856729us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54003b74d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:41.271729 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4201392896}
W20260709 13:20:41.263553 23401 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:20:41.296958 23161 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.298815 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.233981 23396 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:20:41.288448 23160 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.307407 23170 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:20:41.309837 23367 consensus_meta.cc:287] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.004s
I20260709 13:20:41.310205 23040 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:41.313529 22537 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.246786 23398 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:20:41.238690 16561 connection.cc:256] RPC call timeout handler was delayed by -2.4172766970000339s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
I20260709 13:20:41.316956 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:41.331528 16561 connection.cc:256] RPC call timeout handler was delayed by -2.0671766969999226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 13:20:41.331738 17414 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
W20260709 13:20:41.332031 16561 connection.cc:256] RPC call timeout handler was delayed by -2.0005766970000423s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
W20260709 13:20:41.332365 16561 connection.cc:256] RPC call timeout handler was delayed by -1.9159766970001328s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
I20260709 13:20:41.332846 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.317678 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.314849 16667 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:41.318378 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.334612 17420 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:41.335613 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.336195 17420 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:41.337056 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.337736 17420 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:41.272625 23323 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.339452 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:41.332887 16561 connection.cc:256] RPC call timeout handler was delayed by -1.8367766970000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
W20260709 13:20:41.340317 16561 connection.cc:256] RPC call timeout handler was delayed by -1.7738999210000657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1696s for a response
I20260709 13:20:41.285266 23335 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:41.343756 17411 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 8 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:41.327025 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:41.348908 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:41.349468 17410 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:41.340636 16561 connection.cc:256] RPC call timeout handler was delayed by -1.6559999210001024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1827s for a response
I20260709 13:20:41.326330 23170 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:41.347733 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.359352 17420 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:41.247258 23318 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:41.350359 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4282999209999616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
I20260709 13:20:41.356169 23160 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.357730 17410 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:41.329387 23308 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.362077 22947 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:41.324937 23375 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:20:41.329425 23040 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:41.331398 23398 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:41.364838 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4282999209999616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
I20260709 13:20:41.320792 23266 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:41.371140 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.354507 23004 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:41.381047 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2718724290000409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 13:20:41.332512 23367 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:41.388146 16561 connection.cc:256] RPC call timeout handler was delayed by -1.2718724290000409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 13:20:41.393218 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1997315179999077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:20:41.393781 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1944724290001432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:20:41.394275 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1944724290001432s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20310000000000003s for a response
W20260709 13:20:41.394569 16561 connection.cc:256] RPC call timeout handler was delayed by -1.1726999210000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2364s for a response
W20260709 13:20:41.394902 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0936724290002076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 13:20:41.395195 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0936724290002076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 13:20:41.395463 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0540724290001435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
W20260709 13:20:41.395852 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0540724290001435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 13:20:41.396727 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.385960 23266 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:41.393162 23407 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:20:41.343103 23335 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:41.386751 23328 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:41.399750 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.387027 23388 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:41.387177 23398 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:41.402776 23109 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:41.407459 23004 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:41.416633 23314 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:41.439025 23342 consensus_meta.cc:287] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260709 13:20:41.439507 23342 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.440534 17416 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:41.441064 23332 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:41.408299 23367 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:41.407939 23040 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:41.429186 23328 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:41.443778 23040 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:41.444231 23328 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.445528 23328 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.408731 23170 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.447402 23170 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.450811 23160 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:41.451288 23160 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.451650 23160 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:41.372119 23308 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.453763 23308 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:41.409749 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.414033 23323 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:41.462811 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 53172.7us
I20260709 13:20:41.459296 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:41.464368 16588 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
W20260709 13:20:41.427525 17415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 1023) took 2206 ms (client timeout 2116 ms). Trace:
W20260709 13:20:41.465646 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.428260 17413 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 19. Current term is 20. Ops: []
W20260709 13:20:41.428417 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.421947 23335 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.405161 23299 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.455658 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:41.460153 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:41.462118 16593 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 13:20:41.465529 17415 rpcz_store.cc:269] 0709 13:20:39.221405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.357836 (+136431us) service_pool.cc:224] Handling call
0709 13:20:41.427316 (+2069480us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54004b6254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:41.427454 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2559161600}
I20260709 13:20:41.474318 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.475347 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.477455 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.486522 16664 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:41.478374 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.488518 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:41.489460 16559 connection.cc:256] RPC call timeout handler was delayed by -1.1642026169997735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:20:41.490293 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.482592 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.491938 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.495148 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.496486 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.498373 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:41.498536 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 98863.8us
I20260709 13:20:41.500301 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:41.495793 16671 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:20:41.435559 23109 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.505896 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.507278 23420 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:41.510776 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.511224 16671 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:20:41.508389 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.520388 16740 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:41.521013 16742 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:41.483757 23305 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:41.512616 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.523707 16743 raft_consensus.cc:2410] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:20:41.524418 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.525346 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.525974 16737 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:41.527279 23305 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:41.527869 23342 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:41.512333 23109 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:41.516826 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.510809 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:41.519441 16596 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 1275) took 2816 ms (client timeout 1620 ms). Trace:
I20260709 13:20:41.504588 23388 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:41.534155 23388 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:41.530714 23411 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:41.532511 16738 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
W20260709 13:20:41.532568 16596 rpcz_store.cc:269] 0709 13:20:38.702891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:38.871029 (+168138us) service_pool.cc:224] Handling call
0709 13:20:41.515823 (+2644794us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540023afd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:41.519373 (+  3550us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3486127360}
I20260709 13:20:41.528301 23305 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.516341 16590 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:20:41.540962 16590 pending_rounds.cc:85] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 14
I20260709 13:20:41.528144 23323 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:41.545305 23171 consensus_queue.cc:1048] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:41.546586 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1391) took 1105 ms. Trace:
I20260709 13:20:41.547017 17413 rpcz_store.cc:276] 0709 13:20:40.441207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.441384 (+   177us) service_pool.cc:224] Handling call
0709 13:20:41.428073 (+986689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:41.466762 (+ 38689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.546369 (+ 79607us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54004b6250. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:41.546519 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2268342016}
I20260709 13:20:41.537258 22537 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:41.554416 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:41.555716 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.541245 16590 pending_rounds.cc:107] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:20:41.557946 16590 raft_consensus.cc:2889] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:41.536566 16668 raft_consensus.cc:2410] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:20:41.539870 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.542985 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.559111 16737 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:41.561061 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.562170 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.563400 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1389) took 2475 ms. Trace:
I20260709 13:20:41.563795 17412 rpcz_store.cc:276] 0709 13:20:39.081487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.081601 (+   114us) service_pool.cc:224] Handling call
0709 13:20:39.081881 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:39.186147 (+104266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:39.186197 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:39.186257 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:39.186308 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.556428 (+2370120us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400567054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:41.556473 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:41.556582 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239164160}
I20260709 13:20:41.565708 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.568898 16738 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 22: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:41.563529 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.573447 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:41.591610 23189 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:41.579692 23398 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:41.592146 23189 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:41.592622 23189 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:41.593853 23398 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:41.580637 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.594480 23388 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:41.597013 16745 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.599505 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:41.600565 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:41.605525 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:41.607280 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 97558.5us
I20260709 13:20:41.607156 16741 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:41.587222 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.588119 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.588804 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.602190 23335 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.610359 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:41.612164 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90347.6us
I20260709 13:20:41.614195 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.584196 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.617422 16736 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.589622 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.618827 16738 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:41.608825 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:41.621325 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 89740us
I20260709 13:20:41.626565 16666 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:41.567363 22873 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.642021 16745 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:41.575167 23422 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.571477 23314 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:41.606237 23158 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:41.649526 22707 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:20:41.664989 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:41.675904 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61770.6us
I20260709 13:20:41.653494 23422 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:41.678043 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:41.678472 23422 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.652964 23109 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.613917 23189 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.679718 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:41.679951 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 70419.7us
I20260709 13:20:41.681125 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.710690 23308 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:20:41.740077 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.741285 16736 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 12.
I20260709 13:20:41.679742 23266 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:41.674829 16745 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37636 (request call id 2628) took 4862 ms (client timeout 1959 ms). Trace:
I20260709 13:20:41.711227 23429 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:41.648813 23421 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:41.744762 16745 rpcz_store.cc:269] 0709 13:20:36.787871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:36.788734 (+   863us) service_pool.cc:224] Handling call
0709 13:20:38.981142 (+2192408us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4c0002d800. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327deb0a20 00007f327dec1a29 00007f327de87644 00007f327deaef2d 00007f327deaeb93 00007f327fbdf18b 00007f327fbdfcb5 00007f327fbdaf30 00007f327fbdc7de 00007f327fbe827c 00007f3284ba9b17
0709 13:20:41.596348 (+2615206us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b58003725d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe827c 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c
0709 13:20:41.649647 (+ 53299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3557601280}
I20260709 13:20:41.746672 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.674355 23292 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.750066 16740 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 12: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:41.674060 23158 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:41.712754 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:41.713052 23343 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.713534 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:41.759806 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47140.2us
W20260709 13:20:41.760274 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46826.6us
I20260709 13:20:41.761018 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.714499 23318 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.715389 23423 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:41.716907 23335 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:41.717549 16744 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1630) took 2374 ms. Trace:
I20260709 13:20:41.720764 17376 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:41.721343 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 120825us
I20260709 13:20:41.785784 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.647727 22873 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.790235 22873 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.794123 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:41.795941 23109 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.796641 23189 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.797885 16670 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:20:41.798444 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79590.4us
I20260709 13:20:41.718175 22853 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:41.813077 23305 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:41.813623 23335 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:41.716902 23111 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.814851 23344 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:41.815330 23344 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:41.816198 23344 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.816860 23410 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:41.817490 23344 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.818066 23410 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:41.818490 23410 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:41.672511 23314 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:41.819134 23314 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.814738 23335 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:41.820288 23335 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:41.821313 23335 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.761471 23158 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:41.825929 16743 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
I20260709 13:20:41.678761 23194 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.828526 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.831127 23426 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:41.832118 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.834285 17412 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:41.835614 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.835915 17413 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:41.837210 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:41.837299 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4630) took 3901 ms. Trace:
I20260709 13:20:41.837733 16590 rpcz_store.cc:276] 0709 13:20:37.935854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.449340 (+1513486us) service_pool.cc:224] Handling call
0709 13:20:41.515981 (+2066641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:41.558422 (+ 42441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.837110 (+278688us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b540023afd0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:41.837245 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4751390464}
I20260709 13:20:41.839293 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 1099) took 1235 ms. Trace:
I20260709 13:20:41.839989 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.840557 23302 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:41.840690 17416 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:41.841542 23302 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:41.842064 23302 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:41.842975 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.842936 23302 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:41.846405 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:41.847676 23004 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:41.848107 23004 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:41.848912 23004 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.850162 23004 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:41.839743 17409 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:41.839694 16665 rpcz_store.cc:276] 0709 13:20:40.603394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.603512 (+   118us) service_pool.cc:224] Handling call
0709 13:20:40.603786 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.795942 (+192156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:40.795986 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:40.796056 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:40.796114 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.839035 (+1042921us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540008be54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:41.839084 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:41.839234 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441347968}
I20260709 13:20:41.854161 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.854331 23410 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.858566 23323 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:41.860179 23323 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:41.860625 23323 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:41.860365 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.869019 17413 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:41.770079 23160 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.771287 16663 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:41.720422 23342 consensus_queue.cc:1048] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:20:41.744822 23266 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:41.785948 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.680377 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:41.786273 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:41.769330 23422 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.786545 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:41.799093 23416 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:41.801146 23368 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.803192 23428 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:41.803452 22929 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:41.762125 23425 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:41.762543 23408 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:41.761934 23292 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.780429 23423 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.822700 23335 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:40.802335 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:41.782003 16744 rpcz_store.cc:276] 0709 13:20:39.343580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.343680 (+   100us) service_pool.cc:224] Handling call
0709 13:20:39.343952 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:40.460908 (+1116956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.717289 (+1256381us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400433d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:41.717464 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2519305472}
I20260709 13:20:41.916528 23160 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.744329 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:41.875140 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 92550.4us
I20260709 13:20:41.876827 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.921726 16736 raft_consensus.cc:2393] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:41.881109 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.887225 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4213) took 2488 ms. Trace:
I20260709 13:20:41.922385 16739 rpcz_store.cc:276] 0709 13:20:39.399222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:39.403743 (+  4521us) service_pool.cc:224] Handling call
0709 13:20:39.403975 (+   232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:40.438640 (+1034665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:41.886969 (+1448329us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540007c6d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:41.887160 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2378142976}
I20260709 13:20:41.855607 23410 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:41.876295 23345 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:41.891965 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:41.891723 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.893193 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 334) took 8805 ms. Trace:
I20260709 13:20:41.893220 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:41.894075 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:41.928220 16743 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:41.894847 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.898243 23428 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:41.874697 23336 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:41.900072 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:41.861696 23323 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.890643 23213 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:41.875381 23305 consensus_queue.cc:1175] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 14 status { last_received { term: 14 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 14 index: 11 } }
I20260709 13:20:41.887210 23266 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:41.904474 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:41.891728 23408 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:41.908483 23423 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:41.877336 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 91156.1us
I20260709 13:20:41.976506 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:41.954125 23213 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.981815 23213 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.984107 16738 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:41.932854 23345 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:41.989173 23345 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.989616 23292 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:41.995132 16491 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 13, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 13 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:41.887323 23189 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:41.897187 16710 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:41.935207 16636 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:41.919350 22929 consensus_queue.cc:1243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:20:41.957088 16588 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:41.957127 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet e8b0d179ad714b59b34ab1dc57a57c02 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 13) from {username='slave'} at 127.0.0.1:52804
I20260709 13:20:42.010788 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:42.011723 16669 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:41.960872 23408 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:41.935948 22962 consensus_queue.cc:1059] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:20:42.017705 16738 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:42.018783 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36082 (request call id 3634) took 1065 ms. Trace:
I20260709 13:20:42.019416 16663 rpcz_store.cc:276] 0709 13:20:40.953243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.953357 (+   114us) service_pool.cc:224] Handling call
0709 13:20:42.018515 (+1065158us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54000f37d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:42.018687 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849937792}
I20260709 13:20:42.002195 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:42.002544 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.003732 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:41.933483 23428 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:42.023123 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.023499 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.024171 16594 raft_consensus.cc:2376] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22. Re-sending same reply.
W20260709 13:20:42.024959 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 295832us
I20260709 13:20:42.004228 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.005856 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.006068 16743 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:42.026539 16595 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:20:42.010838 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75748us
I20260709 13:20:41.961385 23424 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:20:41.997603 16491 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 16, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 16 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:42.032405 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:42.032564 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 135477us
I20260709 13:20:42.027108 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:41.931674 16492 rpcz_store.cc:276] 0709 13:20:33.083405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.088725 (+  5320us) service_pool.cc:224] Handling call
0709 13:20:33.433828 (+345103us) spinlock_profiling.cc:243] Waited 79.1 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:20:41.888854 (+8455026us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:37.264224 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:37.264990 (+   766us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:37.265038 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:20:37.265064 (+    26us) tablet.cc:662] Decoding operations
0709 13:20:37.277021 (+ 11957us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:37.288534 (+ 11513us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:37.289521 (+   987us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:20:37.321882 (+ 32361us) tablet.cc:701] Row locks acquired
0709 13:20:37.321901 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:20:37.322050 (+   149us) write_op.cc:270] Start()
0709 13:20:37.322164 (+   114us) write_op.cc:276] Timestamp: P: 1783603237321998 usec, L: 0
0709 13:20:37.322187 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:20:37.322854 (+   667us) log.cc:844] Serialized 2059 byte log entry
0709 13:20:38.071281 (+748427us) op_driver.cc:464] REPLICATION: finished
0709 13:20:38.219443 (+148162us) write_op.cc:301] APPLY: starting
0709 13:20:38.221743 (+  2300us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:38.225861 (+  4118us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:38.225979 (+   118us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:38.938367 (+712388us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:38.961937 (+ 23570us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:20:38.968809 (+  6872us) write_op.cc:312] APPLY: finished
0709 13:20:39.002781 (+ 33972us) log.cc:844] Serialized 57 byte log entry
0709 13:20:39.182039 (+179258us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:39.205747 (+ 23708us) write_op.cc:454] Released schema lock
0709 13:20:39.219862 (+ 14115us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:39.220368 (+   506us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:39.220403 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8660,"reported_tablets":10,"spinlock_wait_cycles":181854592,"child_traces":[["op",{"apply.queue_time_us":130402,"apply.run_cpu_time_us":119731,"apply.run_wall_time_us":1012409,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":626573,"prepare.run_cpu_time_us":12056,"prepare.run_wall_time_us":457976,"raft.queue_time_us":170804,"raft.run_cpu_time_us":10832,"raft.run_wall_time_us":90669,"replication_time_us":748950,"thread_start_us":1059987,"threads_started":5,"wal-append.queue_time_us":779540,"wal-append.run_cpu_time_us":24580,"wal-append.run_wall_time_us":2205860}]]}
I20260709 13:20:42.033954 22685 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:20:42.036826 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.035852 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:42.043529 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:42.043867 17415 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:42.049365 16710 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:42.017457 23323 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:42.043442 22685 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:42.051167 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.051375 16710 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.051798 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.056835 16591 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:20:42.057329 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.057777 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.053484 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.061187 16594 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.015702 23179 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.058470 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:42.023828 23420 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:42.054453 22685 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.043119 23299 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.060431 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:20:42.069113 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.059127 23370 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.016175 23334 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:20:42.075515 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.069300 22685 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.063374 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.050683 23189 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:42.077181 16559 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:42.077334 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.071514 23342 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.056363 23292 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:42.083927 23431 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:42.085623 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.087257 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.091161 23314 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.091859 22853 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:42.046969 22929 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:20:42.094106 16666 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:42.096124 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.100514 16594 raft_consensus.cc:2393] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.055192 23213 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:20:42.074497 23318 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:42.082793 23111 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:42.103753 22685 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:42.106590 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.062788 23040 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:20:42.087356 23189 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:42.093468 23431 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.110392 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.089003 23292 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:42.121656 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.125535 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.127251 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:42.127612 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.129522 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:42.130293 17416 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:42.131238 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:42.133006 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:42.134142 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:42.079878 23158 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.140755 16588 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.130385 23416 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:42.145347 23267 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:20:42.145828 23267 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.146296 23158 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.146842 23267 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.148145 23267 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:20:42.135521 23421 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:42.113554 23111 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.151218 23111 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.093743 23319 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:20:42.151268 23421 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:42.152498 23319 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:42.152937 23319 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:42.152751 23421 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.153998 23319 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.113930 22853 consensus_queue.cc:1059] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:20:42.139233 17415 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 15: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:42.115136 22873 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.159559 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:42.160456 22873 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.160892 23299 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:42.161266 16591 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.161314 23299 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:42.122263 23431 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:42.161855 23299 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:42.167312 23428 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.168807 23428 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:42.176750 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.176581 23370 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:42.177305 23370 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:42.179137 23302 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:42.181282 23375 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:42.187875 16590 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:42.193253 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.199347 16591 raft_consensus.cc:2393] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 11: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.200809 16666 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:42.198558 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:42.203485 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48252.7us
I20260709 13:20:42.095373 23342 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.117944 23189 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:42.131623 23292 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:42.215286 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.210670 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.218340 22707 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:42.218813 22707 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:42.221149 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:42.219650 22707 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.225536 22707 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.225948 16710 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:42.228482 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.185669 22929 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:42.236516 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.239396 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.178284 23370 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.206560 23422 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.245142 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.214752 23426 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.246523 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.251618 16592 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:42.249485 23345 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:42.253368 23128 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:42.253762 23128 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.254583 23128 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.257673 23128 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:42.261799 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.262663 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.263312 16591 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 17: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.207227 23266 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.274386 16664 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:42.240407 23314 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.244458 23227 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:42.230660 23111 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.297153 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.302943 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.305131 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.309176 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.314287 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.315603 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.266968 23319 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:42.319780 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.321070 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.321895 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.297331 23227 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.324723 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.320386 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:42.332295 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.331418 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.340585 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.341717 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.346012 23428 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:20:42.350806 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.351075 23389 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:42.351554 23389 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.351881 23266 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:42.352031 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.352887 16590 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.352423 23389 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.353670 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.346719 23428 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:42.353937 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.354310 23428 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:42.354785 23428 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:42.354485 16592 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.315294 23314 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:42.312309 23420 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:42.317358 23368 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:42.328351 23227 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.328886 23109 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:42.332374 22873 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.323240 23160 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:42.360356 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.356958 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.360251 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.334766 23375 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:42.359149 22947 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.381430 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.382619 16671 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:42.369493 16666 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:42.385253 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.370672 16668 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:42.386453 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.351084 23194 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:42.372614 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:42.359058 23158 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:42.377306 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.379165 16664 raft_consensus.cc:1240] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:42.387463 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.392954 17411 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:42.391009 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.404090 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.405133 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:42.380029 23435 log.cc:927] Time spent T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:20:42.411193 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.411966 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.414294 16743 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:42.415679 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.425416 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.426149 16592 raft_consensus.cc:2368] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13.
I20260709 13:20:42.427143 16592 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:42.392580 22947 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.395699 23336 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:42.402417 23158 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:42.429412 23158 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:42.430078 22947 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:42.430516 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.395131 23438 tablet_replica.cc:333] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:20:42.433040 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:42.433853 16740 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:20:42.433058 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.435185 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:42.424345 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.416913 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:42.447621 16745 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:42.367765 23431 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.401916 23375 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:42.409083 23342 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:42.469805 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.478588 16593 raft_consensus.cc:1240] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:42.411271 23334 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:20:42.487782 16595 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 23: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.495936 16594 raft_consensus.cc:1240] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:20:42.497246 16594 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:42.499629 23438 raft_consensus.cc:2243] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:20:42.500525 23438 raft_consensus.cc:2272] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:20:42.503427 23370 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:42.491475 22537 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:42.489784 16593 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:42.457410 22947 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:20:42.513028 16743 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:20:42.457034 23336 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:20:42.448573 23299 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:42.520583 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 178936us
I20260709 13:20:42.522048 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.523350 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.525341 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.525982 23111 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:42.523548 23171 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:20:42.527187 23171 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:42.527544 23171 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.528437 23171 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.529690 23171 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.515445 23368 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:42.513917 22947 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:42.546360 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.524237 23160 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:20:42.543861 23442 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:42.572635 23227 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.579612 23431 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.583268 23389 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:42.584129 23299 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:42.588045 23438 ts_tablet_manager.cc:1916] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:20:42.545214 23368 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.546965 22947 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:42.552016 23429 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:42.557760 22873 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:20:42.602576 23445 log.cc:927] Time spent T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:20:42.604346 23440 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:20:42.602993 23158 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:42.605777 23444 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:20:42.606699 23443 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:20:42.607296 23158 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.611644 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.613940 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:42.615680 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:42.616794 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109046us
I20260709 13:20:42.617002 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.618130 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:20:42.625830 17374 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.541s (SENT)
I20260709 13:20:42.631111 16588 raft_consensus.cc:1240] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 22. Current term is 23. Ops: []
W20260709 13:20:42.632756 17374 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.967s (SENT)
I20260709 13:20:42.633296 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.636137 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:42.636850 16669 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:20:42.637072 17374 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.620s (SENT)
I20260709 13:20:42.637508 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.646358 17416 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:42.649883 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.629468 22873 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:20:42.635130 23432 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:20:42.551621 23375 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.668262 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.666512 16669 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:42.670322 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.616255 23431 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.670948 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:42.635620 23111 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:42.672199 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.589669 23292 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.672884 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.645561 23342 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:42.638782 23335 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:42.674811 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.674976 16743 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:42.675689 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.672688 22707 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:20:42.652207 23429 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:42.610832 23428 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.695574 23428 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.678740 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:42.635308 23160 consensus_queue.cc:1059] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:20:42.698726 23334 raft_consensus.cc:3055] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:42.699142 23334 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:42.699940 23334 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.701222 23334 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:42.682458 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.662993 22873 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:42.634560 22853 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:20:42.687623 16742 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 11. Current term is 12. Ops: []
I20260709 13:20:42.690200 23341 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:42.690744 23408 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.003s
I20260709 13:20:42.691365 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.715790 16740 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.714359 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.711938 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.715978 22537 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:20:42.719058 22707 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:42.680912 23109 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:20:42.720290 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.674707 23158 consensus_queue.cc:1059] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:20:42.742151 23285 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:42.742574 23285 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:42.743381 23285 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.745733 23442 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:42.692852 23429 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:42.750228 23429 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:42.748970 23227 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.697225 23189 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.715982 23345 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:42.739667 23342 raft_consensus.cc:2955] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Committing config change with OpId 19.13: config changed from index 10 to 13, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:42.750998 23425 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.710193 22873 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:42.758456 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:42.671231 23368 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.740273 23170 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:20:42.757434 23323 consensus_meta.cc:287] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.697s	user 0.009s	sys 0.003s
I20260709 13:20:42.756618 22929 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:42.703238 23314 consensus_queue.cc:1175] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "170921ba08b74a33995ae7c7686f8109" responder_term: 17 status { last_received { term: 16 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:20:42.761415 16740 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:42.762979 17409 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.758153 23109 consensus_queue.cc:1243] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer 170921ba08b74a33995ae7c7686f8109 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:20:42.765295 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 114126us
I20260709 13:20:42.765894 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:42.718670 23408 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.770452 17415 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:42.764132 23423 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.004s
I20260709 13:20:42.778314 23368 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:42.768489 23170 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:42.776386 23423 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.793581 23423 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.804198 23285 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:42.805536 23408 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:42.766680 23425 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.806671 23170 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.823014 23438 ts_tablet_manager.cc:1929] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.12
I20260709 13:20:42.823698 23438 log.cc:1199] T e8b0d179ad714b59b34ab1dc57a57c02 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/e8b0d179ad714b59b34ab1dc57a57c02
I20260709 13:20:42.825361 23429 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:42.857421 16596 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:20:42.860144 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:42.862013 16587 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:42.868228 16662 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:42.868299 16670 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:42.873368 16667 raft_consensus.cc:606] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.873962 16667 raft_consensus.cc:614] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:42.760159 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:42.879886 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 118070us
I20260709 13:20:42.880537 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:42.881878 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 121675us
I20260709 13:20:42.883461 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.886216 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:42.895227 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:42.776793 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.912937 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:42.913617 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 136894us
I20260709 13:20:42.915928 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.918789 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:42.927250 22707 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:42.932106 23429 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:42.932530 23299 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:42.933545 23299 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.936189 23318 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:42.937244 23318 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:42.932175 23227 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:42.941210 23323 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:42.946941 23299 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.948966 16595 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:42.949180 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:42.950531 17416 raft_consensus.cc:2410] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:20:42.956017 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.957531 23416 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:42.968158 22947 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:42.972210 16740 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:42.977552 23004 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:42.980176 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:42.983428 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:42.983747 16662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38524 (request call id 1108) took 2142 ms (client timeout 1729 ms). Trace:
W20260709 13:20:42.984525 16662 rpcz_store.cc:269] 0709 13:20:40.841323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:40.841500 (+   177us) service_pool.cc:224] Handling call
0709 13:20:42.983470 (+2141970us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54001f7554. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:42.983635 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4643525376}
I20260709 13:20:42.978282 16590 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:42.998843 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:42.999071 23375 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:43.000524 23375 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.001206 23189 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.002807 23292 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.019603 23428 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:43.020118 23428 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.020709 22947 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.027935 23409 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:43.029589 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 359487us
I20260709 13:20:43.034570 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.041411 23411 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:43.041921 23411 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:43.045944 23323 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:43.046669 16745 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:43.048496 23456 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:42.994467 23004 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:43.046967 22929 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:20:43.056164 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 50082us
I20260709 13:20:43.063017 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.063923 16742 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:43.065580 23456 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:20:43.078373 23462 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 13:20:43.078794 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:43.085117 16636 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.088080 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.066426 23040 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.080979 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3651) took 1278 ms. Trace:
I20260709 13:20:43.097854 23040 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.103858 23040 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:42.893879 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet e8b0d179ad714b59b34ab1dc57a57c02 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
W20260709 13:20:43.104777 16480 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 211064us
I20260709 13:20:43.095775 16670 rpcz_store.cc:276] 0709 13:20:41.802699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:41.808543 (+  5844us) service_pool.cc:224] Handling call
0709 13:20:41.809004 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:42.871607 (+1062603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:43.080712 (+209105us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540008b454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:43.080910 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434435328}
I20260709 13:20:43.119333 23411 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.126175 23411 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:43.129070 23425 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.025787 23428 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.131374 23428 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:20:43.143850 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 59568.1us
I20260709 13:20:43.144656 22537 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:43.146929 23341 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:43.152328 16742 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:20:43.158236 17415 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:43.158331 23004 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.161398 23004 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.164541 23004 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.168145 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.168555 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.169400 16740 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:20:43.171017 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.171468 17376 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:43.174101 23323 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:20:43.179386 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 260884us
I20260709 13:20:43.174654 23323 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.181046 23323 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:43.180922 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.182308 23319 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:43.184629 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.186024 16590 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 9.
I20260709 13:20:43.191288 16491 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 232) took 9815 ms. Trace:
I20260709 13:20:43.192093 17409 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:43.197058 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:43.197530 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 163104us
I20260709 13:20:43.198156 16670 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:43.199447 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:43.200242 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.204507 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:43.204917 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:43.210032 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.215235 23456 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.215469 23004 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:20:43.216248 23004 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:43.217274 23004 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.222175 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:43.192162 16491 rpcz_store.cc:276] 0709 13:20:33.375597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:33.379041 (+  3444us) service_pool.cc:224] Handling call
0709 13:20:43.190957 (+9811916us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:42.354831 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:42.420804 (+ 65973us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:42.433766 (+ 12962us) write_op.cc:435] Acquired schema lock
0709 13:20:42.439061 (+  5295us) tablet.cc:662] Decoding operations
0709 13:20:42.514156 (+ 75095us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:42.521873 (+  7717us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:42.525644 (+  3771us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:20:42.560162 (+ 34518us) tablet.cc:701] Row locks acquired
0709 13:20:42.563932 (+  3770us) write_op.cc:260] PREPARE: finished
0709 13:20:42.570511 (+  6579us) write_op.cc:270] Start()
0709 13:20:42.570666 (+   155us) write_op.cc:276] Timestamp: P: 1783603242570445 usec, L: 0
0709 13:20:42.570692 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:20:42.571423 (+   731us) log.cc:844] Serialized 907 byte log entry
0709 13:20:42.780036 (+208613us) op_driver.cc:464] REPLICATION: finished
0709 13:20:42.821893 (+ 41857us) write_op.cc:301] APPLY: starting
0709 13:20:42.821978 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:42.823839 (+  1861us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:42.823939 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:42.825320 (+  1381us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:42.825523 (+   203us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:20:42.825586 (+    63us) write_op.cc:312] APPLY: finished
0709 13:20:42.826203 (+   617us) log.cc:844] Serialized 33 byte log entry
0709 13:20:42.826387 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:42.826496 (+   109us) write_op.cc:454] Released schema lock
0709 13:20:42.826541 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:42.827036 (+   495us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:42.827066 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":269723,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":39952,"apply.run_cpu_time_us":4722,"apply.run_wall_time_us":5345,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12495,"num_ops":2,"prepare.queue_time_us":259959,"prepare.run_cpu_time_us":6091,"prepare.run_wall_time_us":307209,"raft.queue_time_us":30345,"raft.run_cpu_time_us":23455,"raft.run_wall_time_us":32350,"replication_time_us":209193,"thread_start_us":304039,"threads_started":4,"wal-append.queue_time_us":130893}]]}
I20260709 13:20:43.230571 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:43.207959 22873 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.243244 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.260468 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:43.264313 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64224.6us
I20260709 13:20:43.250383 23341 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:43.261991 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:43.276925 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43519.3us
I20260709 13:20:43.277220 23285 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:43.277834 23285 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.279081 23285 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.283725 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.284693 22873 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.285636 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.287606 16745 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:43.288373 17415 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:43.290030 16745 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:43.290953 23421 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:43.293766 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.294400 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:43.294924 23170 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:20:43.304258 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:43.302969 23319 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:20:43.307073 22929 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:43.307461 22929 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:43.308408 22929 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.309827 22929 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.311932 17415 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:43.314544 23319 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:43.317451 23299 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:43.317890 23299 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:43.318833 23299 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.319468 16589 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:43.320389 23299 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:43.327373 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.335011 16742 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:43.335388 23389 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.336001 23456 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:43.336051 23389 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.339357 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:43.341240 23368 consensus_meta.cc:287] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.008s
I20260709 13:20:43.342016 16662 raft_consensus.cc:606] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.342535 16662 raft_consensus.cc:614] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:43.342540 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 127963us
I20260709 13:20:43.348253 16662 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:43.349082 16662 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.348486 23375 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:43.352030 23421 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.357199 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:20:43.341975 23368 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:20:43.357802 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.358088 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.367925 22873 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:20:43.377699 16591 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:20:43.372694 23389 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.374342 23302 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:43.391148 23302 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:43.392151 23302 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.393589 23302 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:43.395084 23448 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:43.395162 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.369138 23341 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:43.396106 16589 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:43.396025 23389 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:20:43.358412 23323 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.398197 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:43.382290 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.401266 16670 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:43.381783 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:43.375115 17374 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.372620 23335 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:43.395737 23448 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.407295 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:43.407615 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.407860 23448 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.409731 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.410383 16739 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:43.412065 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.409673 16708 leader_election.cc:400] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13.
I20260709 13:20:43.413329 16708 leader_election.cc:403] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:20:43.402824 23411 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:43.404413 23323 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.409772 23448 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.419018 22873 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:43.424913 23319 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 12. Current term is 13.
I20260709 13:20:43.425452 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.426173 16737 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:43.418849 23335 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:43.418828 22929 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:43.415023 23341 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:43.432315 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:43.433143 22873 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:43.433638 22873 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.434698 22873 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.436331 16742 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:43.418994 23411 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.444187 23411 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.464165 16741 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:43.460968 16595 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:20:43.472249 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97223.1us
I20260709 13:20:43.474233 16739 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
W20260709 13:20:43.475701 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66204us
I20260709 13:20:43.463488 23428 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:43.480786 17376 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:43.491318 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.486555 23421 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:43.492583 23335 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:43.504773 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.492751 23285 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:20:43.505659 23323 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:43.515038 23431 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
W20260709 13:20:43.509670 23476 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:20:43.518651 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.510190 23335 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:43.504005 23428 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:43.525686 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48221.7us
W20260709 13:20:43.526291 23475 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:20:43.533538 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.535753 23335 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.544415 22929 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:43.571390 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.580574 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:43.568082 23341 consensus_queue.cc:1059] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:20:43.580346 23421 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:43.588578 22929 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:43.596297 23319 consensus_queue.cc:1059] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:43.596841 23341 raft_consensus.cc:3055] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:43.598152 23341 raft_consensus.cc:740] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:43.598018 23323 consensus_queue.cc:1059] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:43.599184 23323 raft_consensus.cc:3055] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:20:43.599606 23323 raft_consensus.cc:740] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:43.599426 23341 consensus_queue.cc:260] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.600625 23323 consensus_queue.cc:260] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.601311 23341 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:43.597462 23299 consensus_queue.cc:1059] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:20:43.608179 23473 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.165s	user 0.003s	sys 0.002s
I20260709 13:20:43.612525 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:43.613608 16670 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:43.605095 23323 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:43.616981 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:43.618063 23428 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.618778 23428 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:20:43.623265 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50192.3us
I20260709 13:20:43.627575 23456 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.628397 23456 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.631187 23456 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.632445 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.633203 16745 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:43.634742 17376 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:43.637702 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.603703 23285 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:43.641899 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.642757 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.636709 23456 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:43.647122 23170 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.648761 16745 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:20:43.647614 23480 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.002s
I20260709 13:20:43.639910 23285 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.660059 23482 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:43.648020 23456 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.661324 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:43.673882 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 56997.7us
I20260709 13:20:43.655673 23421 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:43.683822 22537 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:43.688162 23485 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:43.687950 23285 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.673928 23456 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.666265 23482 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:43.686767 22537 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.677088 23389 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:43.652508 23170 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:43.725564 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64337.4us
I20260709 13:20:43.711318 23004 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:43.732215 23425 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.711306 23411 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.732594 23389 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.732857 23425 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.734539 23411 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.735250 23483 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:43.735905 22537 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.736660 23299 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:43.737085 23299 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:43.738005 23299 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.736297 23483 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.708986 23482 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.745767 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:43.746431 16662 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:43.747730 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:43.748647 23170 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.750969 23489 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:43.751475 23489 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:43.751873 23489 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.756170 23483 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.759781 23299 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.756721 23368 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:43.767284 23368 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.767591 23482 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.768390 23285 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.769534 23368 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.770942 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.771667 16745 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:43.774062 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:43.774371 22929 raft_consensus.cc:3055] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:43.774782 22929 raft_consensus.cc:740] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:43.775694 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.775709 22929 consensus_queue.cc:260] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:43.788197 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 111738us
I20260709 13:20:43.780958 23482 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.796671 23485 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.734786 23389 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.798837 23485 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.800981 23481 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:43.801409 23481 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.804798 23485 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.806634 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:43.807403 17409 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:43.807669 22929 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.809046 16745 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:43.802212 23481 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.809887 23456 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.810664 23481 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.734790 23425 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.776988 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.814599 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:43.814509 16670 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:43.815904 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:43.823633 17415 raft_consensus.cc:2410] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 13:20:43.827556 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 113843us
I20260709 13:20:43.831002 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.835377 16492 catalog_manager.cc:5697] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:43.837073 16492 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 8 to 11, term changed from 14 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to <none>, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:43.839283 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:43.839573 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.839249 16492 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:43.840734 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:43.841593 17409 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:20:43.840817 16492 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 17, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:43.842535 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.843250 17410 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:43.842429 16492 catalog_manager.cc:5697] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 17, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:43.843981 23456 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.847045 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.847129 16492 catalog_manager.cc:5697] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:43.848021 16740 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.834589 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:43.849893 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.850446 16739 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.851574 17416 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:43.823199 23485 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:43.828941 23323 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:43.859124 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.832064 23421 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:20:43.860570 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.850013 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:43.865278 16743 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:20:43.862840 17410 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:43.844904 23004 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:43.890729 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:43.891448 16588 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:20:43.871027 23482 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:43.893421 23482 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:43.894435 23482 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.895902 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:43.896764 17409 raft_consensus.cc:2393] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:43.897610 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.901988 23389 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.903015 23302 raft_consensus.cc:993] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:43.903586 23302 raft_consensus.cc:1081] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:43.902714 23389 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.908221 23389 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.911648 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.911935 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.912599 16591 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:43.871654 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.872305 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:43.913517 16595 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.914515 23490 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:43.914772 23389 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.915350 23389 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.872718 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.916718 16741 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:43.917373 23389 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.918853 16670 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:43.919637 16740 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:43.920918 17376 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:43.921696 23389 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:43.847453 23319 raft_consensus.cc:3055] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:43.876608 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.922420 23319 raft_consensus.cc:740] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:43.923429 23319 consensus_queue.cc:260] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.877293 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.924443 23319 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:43.879319 17416 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:20:43.927179 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 92729.7us
I20260709 13:20:43.880182 17411 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:43.880877 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.856992 23485 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:43.883474 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.929822 23485 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:43.884335 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.885193 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.930647 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:43.912509 16590 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
I20260709 13:20:43.921780 23421 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:43.871390 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.926697 23481 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:20:43.845178 23368 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:20:43.931134 17409 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:43.932256 17412 raft_consensus.cc:2410] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 18 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 13:20:43.932598 16591 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:20:43.932879 17413 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:20:43.932399 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:43.879422 23489 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.937270 22537 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:43.945156 16745 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 25.
I20260709 13:20:43.936717 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:43.938781 16739 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:43.947361 16587 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:20:43.948035 16587 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:43.948405 16587 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:43.945371 22537 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:43.955559 22537 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:43.935953 23389 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.955857 16587 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.959353 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:43.938205 16588 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 10.
I20260709 13:20:43.960013 16594 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:43.958170 23389 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.961835 23431 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.962298 23431 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.963256 16737 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:43.963485 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:43.964071 16738 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:43.964391 23431 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.948071 23448 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:43.965503 23448 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:43.967608 23448 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.969362 16587 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:43.853530 16492 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 10 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:43.971868 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:43.944761 16590 pending_rounds.cc:85] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 9
I20260709 13:20:43.972509 16670 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:43.971882 16740 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:20:43.973177 16670 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:43.975960 16595 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:43.971616 23040 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:20:43.977998 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:43.977314 16492 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 20 to 21, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 21 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:43.978647 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:43.977101 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:43.979501 16492 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:43.981037 23425 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.981693 23425 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:43.982734 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:43.981001 16492 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 17, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:43.988545 16587 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:20:43.988818 23425 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.991964 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:43.992172 23285 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:20:43.992547 16662 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:43.997150 22537 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.999002 23389 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:43.999564 23389 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.001569 23389 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:43.989511 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.002343 16743 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:43.945681 23421 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:44.003736 23421 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.003877 16492 catalog_manager.cc:5697] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:44.005851 22537 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.006372 23494 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:43.972874 16590 pending_rounds.cc:107] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:20:44.005191 16492 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 15 to 17, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:44.007090 16590 raft_consensus.cc:2889] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:44.006999 23494 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:43.984125 16670 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.009544 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.012228 17410 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:44.010998 16589 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:44.019199 16738 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:44.019945 16737 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:44.028182 16670 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:44.028358 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.008015 16492 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 15, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 15 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:44.029115 16587 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:44.032435 16492 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: config changed from index 9 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:44.037047 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:44.021131 23285 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:44.038877 16587 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:44.040863 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.048461 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.049146 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.044157 23285 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.039683 23448 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:44.033779 23431 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:20:44.042203 23425 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.049429 16492 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 18 to 20, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 20 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:44.046126 23323 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.067837 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:44.129381 16669 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:20:44.176822 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.067327 16670 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:44.132092 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.138164 16742 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:44.123391 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.099357 16492 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:44.069046 16587 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:44.067971 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:44.187402 16492 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 20 to 24, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 24 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:44.099572 23323 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:44.196722 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 150727us
I20260709 13:20:44.197614 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:44.198675 23483 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:44.199085 23482 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:44.199810 23302 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:44.130898 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.205035 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:44.190395 16589 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:44.200481 23302 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.070638 23285 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:44.071888 23335 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.150162 23389 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.100518 23489 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.152220 23004 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:44.190806 23456 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.070370 23485 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.183738 23448 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:44.216267 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.100692 23425 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.113417 23494 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.247037 16492 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 13 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:44.214777 23481 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:20:44.256572 16740 raft_consensus.cc:2368] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:44.257030 16738 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:44.245751 23448 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:44.251370 23368 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.233203 23389 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.232275 23482 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:20:44.272954 23485 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.279155 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.279822 16667 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:20:44.285158 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.290238 23499 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:20:44.291064 23489 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.291790 23489 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.262702 23456 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.296723 23456 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.298486 23302 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.304991 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.305922 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.308106 23040 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:20:44.308580 23040 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.309046 23040 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:44.311094 23389 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:44.308710 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.311765 23389 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:44.312316 23389 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:44.313210 23389 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.315285 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.316095 23411 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:44.316514 23411 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.316847 23411 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:44.317329 16634 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.318651 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:44.319577 16664 raft_consensus.cc:2410] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:20:44.320544 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:44.320868 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.321684 23170 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:44.322070 23170 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.322438 23170 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:44.324717 23285 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.325418 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.327127 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:44.327975 23425 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.329305 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:44.271152 23368 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.331576 22929 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:44.332453 22929 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:44.332490 23431 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:44.332458 23323 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.335039 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.334554 23040 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.331014 23411 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.335968 23495 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:44.337294 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.331662 23368 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:44.340797 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 180280us
I20260709 13:20:44.341432 16636 leader_election.cc:400] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:44.341754 16636 leader_election.cc:403] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:44.311136 16667 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:44.233003 23335 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.344707 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.339767 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:44.373267 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.373952 16737 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:44.341197 23495 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.382968 23319 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:44.385934 23319 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:44.386951 23319 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.388901 23489 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.389305 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:44.389626 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 132983us
I20260709 13:20:44.389966 16667 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:44.390926 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:44.391141 16669 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:44.392453 16559 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:44.392627 22537 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:44.395200 22537 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.395783 22537 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:44.397938 23285 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:44.399019 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 58514.4us
I20260709 13:20:44.349470 23302 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:44.403995 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.405310 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:44.406039 16669 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:20:44.413224 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.414023 16670 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:44.415306 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.369115 23409 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:44.411361 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.374711 23004 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:44.420336 16743 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:44.402576 23495 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:44.420014 22537 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.432911 16670 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:44.434303 16669 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:20:44.368419 23503 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:20:44.443351 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.444347 17411 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:44.458575 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.459218 16669 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:44.413028 23302 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:20:44.467839 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 128156us
I20260709 13:20:44.468540 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:44.459635 22537 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.419835 23285 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:44.419771 23494 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:44.418394 23489 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:44.411280 23431 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:44.434710 23323 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:20:44.472698 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 81875.6us
W20260709 13:20:44.488821 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:44.491276 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.456480 23040 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.495664 16670 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:44.497244 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:44.497915 16666 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:44.495339 16743 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 13: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:44.502022 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.502852 16587 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:44.511147 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.484341 23494 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:44.515929 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.486040 23323 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.513187 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:44.521955 17414 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:44.510847 16669 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 13.
I20260709 13:20:44.528328 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.527278 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.529902 17409 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:20:44.529968 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:44.532948 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 63442.3us
I20260709 13:20:44.496979 23285 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:44.534163 16738 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:44.497305 23456 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:44.473023 23004 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.529592 23323 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:44.535207 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.528048 23494 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:44.535045 17376 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.533847 23489 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:44.540727 23040 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.548030 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.543586 23285 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.546159 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.557765 16740 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:44.547863 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:44.558712 16595 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:20:44.543699 23456 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.555339 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:44.560638 16737 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:20:44.562067 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.560253 16590 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:44.560110 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:44.567487 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.557473 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:44.572316 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.544888 23517 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.537431 23170 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.577147 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.558269 23319 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.567413 23040 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.582279 23523 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:44.582893 23523 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.583235 23523 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:44.584280 23040 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.584947 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.585687 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.598191 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:44.587172 23335 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.587234 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:44.588449 16669 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:44.598898 16587 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:44.578087 17414 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:44.561249 23411 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.596066 16737 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 26.
I20260709 13:20:44.598235 17415 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:44.601541 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:44.605412 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.605681 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:44.606207 17416 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:44.606015 17410 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:20:44.615046 16595 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 26.
I20260709 13:20:44.572953 23368 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:44.620198 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.616354 16594 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:44.616902 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.621811 17411 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:44.617589 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.623006 16738 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:20:44.623083 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.620826 17414 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:44.628304 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.628866 17410 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:44.552255 23302 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.638757 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.645090 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.647178 17414 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:44.652127 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:44.652892 17419 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:44.610230 23517 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.661194 23517 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.661815 23508 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:20:44.642582 23489 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.663343 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.664947 23525 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:44.665378 23525 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.665758 23525 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:44.666169 23519 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:20:44.596908 23319 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.668633 23319 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:44.570307 23431 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.650678 17411 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:44.654729 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:44.650750 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.675618 16594 raft_consensus.cc:1240] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
W20260709 13:20:44.678900 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 89322.7us
I20260709 13:20:44.685962 17416 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
W20260709 13:20:44.688480 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 111923us
I20260709 13:20:44.693638 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:44.696941 16743 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:44.697691 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:44.703438 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:44.709295 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:44.710114 17410 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:44.714390 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.715708 17414 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:20:44.721333 17410 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:44.728201 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.678529 23511 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:44.688764 22873 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:44.753237 22873 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.753854 22873 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:44.638665 23368 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:44.747485 16561 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.4:41455 timed out after 1.765s (SENT) [suppressed 32 similar messages]
I20260709 13:20:44.678664 23508 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:20:44.695892 23483 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:20:44.702832 23510 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:20:44.766088 16666 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:44.779958 17419 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:44.774803 16591 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
W20260709 13:20:44.776242 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 168274us
I20260709 13:20:44.743635 22929 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:44.806615 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:44.782976 23448 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.003s
I20260709 13:20:44.814158 16743 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:20:44.705432 23523 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.819912 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:44.779412 23511 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:44.805903 23507 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:20:44.827666 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:44.842671 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 152627us
I20260709 13:20:44.847661 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:44.854009 23527 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:20:44.855020 23518 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:20:44.859658 23456 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:44.887784 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:44.881450 17412 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:44.784721 23323 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:44.899417 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 196653us
I20260709 13:20:44.827550 23448 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.890007 23170 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.917426 17409 raft_consensus.cc:2368] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:20:44.923236 16595 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:20:44.927459 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83785.5us
I20260709 13:20:44.919893 23319 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.917073 23510 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:44.925585 23530 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:44.921846 23170 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:44.935828 23511 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:44.906354 23541 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:44.943320 23368 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:44.954784 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 207365us
W20260709 13:20:44.962739 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:44.962808 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 115266us
I20260709 13:20:44.948381 23170 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.980258 23323 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.979355 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:44.980808 23523 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.947043 23319 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.981467 23302 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:44.949898 23494 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:44.983275 23319 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:44.984601 23524 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.313s	user 0.003s	sys 0.000s
I20260709 13:20:44.985141 23494 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.986291 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:44.987021 23302 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:44.969913 23495 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.987464 23302 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:44.988011 23302 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:44.989300 16710 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:44.951977 23510 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.991953 22873 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:44.992898 23510 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.993265 23531 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:44.993791 23531 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:44.994213 23531 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:44.995566 22873 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.003453 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.043609 16589 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:45.004808 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:45.004912 23448 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.005105 17415 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:45.009127 23004 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:44.953989 23525 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.046263 17419 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:45.038729 23537 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.109s	user 0.000s	sys 0.002s
I20260709 13:20:44.969626 23170 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:45.032455 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.054848 16743 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:45.035248 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49029us
I20260709 13:20:45.035897 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.036648 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.057796 16738 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.057888 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:44.989332 23495 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.058893 16739 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:45.057897 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.060184 22537 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:45.061843 23299 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.062343 23299 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.063243 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.040222 23302 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.064455 23299 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.065121 23528 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.065201 23302 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.065932 23495 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:45.067466 23532 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.138s	user 0.003s	sys 0.000s
I20260709 13:20:45.056979 23490 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:45.068738 23490 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.056124 23485 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.071954 23485 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.073812 23485 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:45.074249 23485 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:45.065665 23528 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.076431 23542 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:45.077848 23528 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.070627 23490 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:45.079195 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:45.084088 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:45.085156 17419 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:45.082068 16738 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:45.086766 17376 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.087231 16664 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:45.088410 16662 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:45.086427 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.080523 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.081378 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:45.082401 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:45.093411 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.096812 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.097527 16670 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:45.092892 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:45.095927 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 194672us
I20260709 13:20:45.094111 16742 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:45.100871 23299 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:45.093971 23004 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:45.101301 23299 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.088830 23503 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:45.101827 23299 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:45.102197 23503 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.101842 23004 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:45.104086 23448 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:45.105623 17376 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.047727 23531 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.108364 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:45.092659 23302 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:45.109174 23335 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:45.121799 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:45.136154 16636 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:45.137480 22537 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:45.138186 22537 raft_consensus.cc:697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:45.117200 23302 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.121143 23335 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:45.143998 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63546.2us
I20260709 13:20:45.144203 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.120373 23421 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.165707 23299 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.166198 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:45.166144 23421 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.167928 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.168787 23448 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:45.169301 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.170472 23448 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:45.170567 23421 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.170945 23514 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:45.104195 23004 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.145125 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.173569 23485 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:45.173909 23530 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.173975 23485 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:45.174722 23485 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:45.174651 23530 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.175462 23335 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:45.176110 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.177346 23530 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.177419 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.178020 23530 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:45.124518 23490 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:20:45.146898 23302 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.146538 23503 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.143936 23522 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:45.187422 23522 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.187533 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.187834 23522 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:45.191092 16594 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:45.191720 23448 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:45.192874 23335 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.154922 22537 consensus_queue.cc:237] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.159010 23554 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:45.160506 23531 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.161641 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.161770 23525 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.165499 23495 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:45.147311 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.219879 16589 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:45.162850 16743 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:45.220368 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.143524 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:45.220961 16596 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:45.221580 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 78177.3us
I20260709 13:20:45.222397 16636 leader_election.cc:400] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:20:45.182747 23482 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:45.225723 23551 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:20:45.183291 23526 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:45.228914 23302 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:45.184283 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.231848 23530 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.232402 23530 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.142081 23529 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:20:45.198716 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:45.234078 23529 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:45.234190 17410 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:45.234411 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.235085 23529 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.236678 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.237310 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:45.239042 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.239763 16743 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:45.241817 22929 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:45.242256 22929 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:45.240782 23522 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.248896 23522 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.250033 16710 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.215282 17413 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:45.256639 22929 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.256852 23485 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.257987 22929 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.259178 23485 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:45.159569 23170 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.261960 23170 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.262733 16666 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:45.264818 23490 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:45.169935 23299 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.249938 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.265196 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:45.265851 16595 raft_consensus.cc:2393] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.265898 17410 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:45.267490 16666 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:20:45.251637 23530 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.268234 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.269274 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.269997 16668 raft_consensus.cc:2368] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:45.252974 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.235050 16742 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:45.253829 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.254290 23302 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.214835 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.272727 16594 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.273155 17413 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:45.267952 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.271333 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.277053 17415 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:45.276947 16589 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:45.194530 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:45.271211 16665 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
W20260709 13:20:45.288501 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 94037us
I20260709 13:20:45.288919 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:45.291923 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.292513 17410 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:45.290601 16708 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:45.276412 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.295889 16667 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:45.296737 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:45.297521 16662 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:45.290649 16710 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.302897 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.303542 17413 raft_consensus.cc:2376] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18. Re-sending same reply.
I20260709 13:20:45.307936 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.308208 17410 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:45.300453 17376 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.265215 23490 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.316330 17412 raft_consensus.cc:2368] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:20:45.323566 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.324184 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.324451 16591 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.325078 17416 raft_consensus.cc:2410] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:20:45.326378 16669 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:45.295832 23448 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:45.333034 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:45.338930 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.334725 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.341017 16667 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:20:45.297343 23530 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:45.301829 23485 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.346939 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.347730 16587 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.302559 23523 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:45.309274 23531 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:45.328353 23503 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:45.353634 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:45.359972 16742 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:20:45.354318 16596 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:45.358206 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:45.326756 23490 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:45.362692 16663 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:45.362795 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.363571 16743 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:45.361514 17376 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.371214 16591 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:45.345537 23530 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:45.388036 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64884.6us
I20260709 13:20:45.388922 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.391364 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.392678 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.349838 23319 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:45.393682 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.401373 16594 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.333344 23495 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:45.402762 16591 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:20:45.398308 16634 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:45.346632 23551 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:45.400449 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109933us
I20260709 13:20:45.364445 23448 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:45.408324 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.402107 16743 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:45.375099 23523 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:45.407665 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.417268 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.418237 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.421985 16741 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:45.422183 16594 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:45.417797 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.376307 23544 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.365716 23503 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:45.418913 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.419678 16595 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:45.379142 23531 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:45.420490 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.423146 16596 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:45.401749 23530 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.350521 23485 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.404227 23483 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:45.405130 23421 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:45.427652 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.380103 23425 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:20:45.441184 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.440518 16634 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:45.443683 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.444396 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.446312 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.420778 23319 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:45.427925 23503 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:45.453145 16590 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:45.459313 16593 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.428046 23302 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:45.456717 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.461129 16588 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:45.469521 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:45.470461 16634 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:45.471004 16596 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:45.493109 16741 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:45.440702 23483 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.499516 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.500254 16591 raft_consensus.cc:2368] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:45.409502 23495 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.417411 23551 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.497951 16743 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:45.457693 22873 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:45.510766 23569 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:45.471936 23421 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:20:45.502331 23557 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:20:45.427477 23544 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.502898 23483 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:45.518438 23567 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:20:45.518600 22873 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:45.517453 23562 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:20:45.526033 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:45.504498 23501 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:45.533962 23523 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.535482 23299 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.535542 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.568784 16741 raft_consensus.cc:2393] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 17: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:45.539812 23485 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:45.544021 23561 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:20:45.547705 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 77302.9us
I20260709 13:20:45.502153 23302 raft_consensus.cc:697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:45.511905 23335 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:45.529036 23560 log.cc:927] Time spent T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
I20260709 13:20:45.556951 23522 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:45.523573 23319 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.576633 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.571077 23501 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:45.579917 23501 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.563086 16662 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:20:45.587193 16736 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:45.590214 16739 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:20:45.583177 23170 raft_consensus.cc:2764] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:20:45.595809 16589 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:45.599001 16594 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:45.508203 23531 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:45.671267 23574 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:20:45.584000 23299 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.858438 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.859134 16739 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:45.859441 23299 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:45.710583 23569 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:20:45.748819 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 341250us
I20260709 13:20:45.691792 23421 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.860683 23569 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.862253 16708 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:45.865257 22873 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.866247 23495 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.869882 23503 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.526293 23448 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.874112 16743 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:45.874580 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.875382 16739 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:45.879235 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:45.889407 16743 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:45.890087 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.890897 16592 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:45.891764 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.892263 16592 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:45.893342 16592 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:20:45.895143 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.895792 16587 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:45.869876 23530 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.898999 23530 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.902216 23528 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:45.903301 23323 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:45.903929 23323 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.906188 23323 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:45.907187 23573 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:45.907326 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 45133.3us
I20260709 13:20:45.907752 23573 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.908177 23573 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:45.908289 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:45.917846 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:45.919309 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.919546 16736 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:20:45.920138 23531 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:45.920823 23531 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:45.835433 23483 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.923055 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.923718 16666 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:20:45.924415 16595 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.924657 23483 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.925841 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.835968 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.837275 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.926458 23503 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.927011 16594 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:45.837502 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:45.928193 16589 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:20:45.926483 16662 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:45.929889 16596 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:45.931540 16663 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:45.932669 16669 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:45.937810 16596 raft_consensus.cc:3055] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:45.938226 16596 raft_consensus.cc:740] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:45.938926 16596 consensus_queue.cc:260] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.939846 16596 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.940436 23573 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.942783 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.679235 23302 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.632606 23389 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
W20260709 13:20:45.951200 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49264.3us
I20260709 13:20:45.854606 23551 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.618441 23522 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:45.832298 23544 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:45.952529 23569 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:45.978466 23569 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.978924 23569 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:45.978631 23522 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.953207 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.953542 23573 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:45.853924 23490 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.955869 23421 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.957124 23004 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:45.957614 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.986141 17415 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.957949 23542 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:45.987217 23542 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:45.958635 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:45.963305 23510 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:45.988595 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:45.989470 16738 raft_consensus.cc:2368] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:45.964550 17374 leader_election.cc:400] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:45.990060 17374 leader_election.cc:403] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:45.988942 17416 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:45.993284 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:45.969681 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.976994 16596 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:20:45.988451 23573 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:45.996641 23573 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:45.994707 23319 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.997239 23573 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:45.997449 23319 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:45.998961 23572 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:45.999559 23572 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.000049 23319 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.001830 23572 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:46.001698 16491 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 19 to 22, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 22 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:45.982755 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:46.002911 23483 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:46.032007 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:45.979832 23489 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:20:46.038247 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55537.8us
I20260709 13:20:46.004679 16491 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:46.039650 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:46.008810 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:45.980302 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61055.2us
I20260709 13:20:46.033609 16663 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:46.043556 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.043776 16669 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:46.041900 16491 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 9 to 12, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 12 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:46.041926 17410 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:46.026207 17415 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:46.044910 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:46.045289 16491 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 18 to 20, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 20 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:46.046377 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.042327 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.058712 16491 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 14, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 14 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:46.063175 17410 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:20:46.072001 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.072887 17410 raft_consensus.cc:2393] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:46.074712 16710 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:45.995267 16596 pending_rounds.cc:85] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 12
I20260709 13:20:46.082755 16596 pending_rounds.cc:107] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:20:46.083489 23517 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:46.084064 16596 raft_consensus.cc:2889] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:46.087742 23040 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:46.088855 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:46.060001 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:45.994526 23421 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.091593 23411 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:46.092025 23411 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.092374 23411 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:46.093904 23421 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:46.094552 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.095273 17411 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:46.073626 23551 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:46.096524 23551 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:46.074433 16491 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 14 to 16, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 16 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:46.096966 23551 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:46.097586 16491 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:46.100821 23335 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:46.036599 23523 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:46.101352 23335 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.036404 23510 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:20:46.108670 17409 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:46.110271 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.112425 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.118664 17420 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:46.115202 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:20:46.123562 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 83381.8us
I20260709 13:20:46.126674 17416 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:46.100132 16491 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:46.129448 17411 raft_consensus.cc:1275] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:46.129992 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.130625 17410 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:46.131294 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 335) took 2564 ms. Trace:
I20260709 13:20:46.136775 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.139935 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:46.138238 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:46.141793 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:46.143409 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:46.146855 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.147804 16589 raft_consensus.cc:3060] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:46.111106 23586 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:20:46.166215 17413 raft_consensus.cc:1240] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:46.083134 23448 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:46.154403 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:46.156335 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.160641 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.171640 16594 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:46.171763 16588 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:46.078596 23530 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:46.032009 22873 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.111s
I20260709 13:20:46.169274 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.179798 16590 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:46.149025 23550 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:20:46.149340 23529 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:20:46.204093 23550 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:46.204509 23550 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:46.173961 16591 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:46.137252 23577 raft_consensus.cc:2804] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:46.205379 23550 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.206233 23577 raft_consensus.cc:697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:46.166074 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.207365 16593 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:20:45.994347 23004 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.207397 23577 consensus_queue.cc:237] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.209419 23004 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.128032 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:46.052206 23528 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.110s
W20260709 13:20:46.210626 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 82722.1us
I20260709 13:20:46.211706 23514 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:46.131867 16492 rpcz_store.cc:276] 0709 13:20:43.513475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:43.513788 (+   313us) service_pool.cc:224] Handling call
0709 13:20:46.077919 (+2564131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:44.542260 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:44.663370 (+121110us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:44.663423 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:20:44.663440 (+    17us) tablet.cc:662] Decoding operations
0709 13:20:44.665084 (+  1644us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:44.665161 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:44.665191 (+    30us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:20:44.666781 (+  1590us) tablet.cc:701] Row locks acquired
0709 13:20:44.666835 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:20:44.729156 (+ 62321us) write_op.cc:270] Start()
0709 13:20:44.747043 (+ 17887us) write_op.cc:276] Timestamp: P: 1783603244697544 usec, L: 0
0709 13:20:44.747084 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:20:44.851954 (+104870us) log.cc:844] Serialized 7119 byte log entry
0709 13:20:45.078441 (+226487us) op_driver.cc:464] REPLICATION: finished
0709 13:20:45.171525 (+ 93084us) write_op.cc:301] APPLY: starting
0709 13:20:45.171634 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:45.228063 (+ 56429us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe132a 00007f327fb66f14 00007f327fb66ec3 00007f327fb66e38 00007f327fb66df3 00007f327fb660dc 00007f327fb8fe16 00007f327fb77cbb 00007f327fb6239d 00007f327fb65d75
0709 13:20:45.349625 (+121562us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:45.349678 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:45.919009 (+569331us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b5c0028fd0c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f3285dd6331 00007f3285dd6221 00007f3285dd0f6c 00007f328353130b 00007f3283530fd3 00007f328362fe34 00007f328361b97c 00007f328361fd61 00007f328361fd19 00007f328361fca9 00007f328361fc71
0709 13:20:45.922565 (+  3556us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:45.929489 (+  6924us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:20:45.929537 (+    48us) write_op.cc:312] APPLY: finished
0709 13:20:45.930000 (+   463us) log.cc:844] Serialized 164 byte log entry
0709 13:20:45.932700 (+  2700us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:45.932878 (+   178us) write_op.cc:454] Released schema lock
0709 13:20:45.932950 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:45.933535 (+   585us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:45.933563 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":104,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":92828,"apply.run_cpu_time_us":64026,"apply.run_wall_time_us":761970,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79439,"num_ops":18,"prepare.queue_time_us":210376,"prepare.run_cpu_time_us":22388,"prepare.run_wall_time_us":431458,"raft.queue_time_us":17344,"raft.run_cpu_time_us":19483,"raft.run_wall_time_us":166520,"replication_time_us":331147,"spinlock_wait_cycles":127370368,"thread_start_us":298886,"threads_started":4,"wal-append.queue_time_us":184282}]]}
I20260709 13:20:46.212158 23514 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.212646 23514 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:46.214700 16710 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:46.216325 23544 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:46.219384 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:46.222422 16589 raft_consensus.cc:2468] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:46.225621 17420 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:46.182832 16594 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:46.179848 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.183409 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:46.184389 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.239584 16587 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:46.165911 23516 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:46.142514 23541 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:46.169757 23494 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:20:46.189248 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.195315 23483 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:46.195616 23542 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.197832 23490 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:46.180178 23530 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:46.258272 17410 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:46.270959 17374 leader_election.cc:400] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:46.284509 16710 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:46.285725 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:46.206681 23550 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:46.245651 23531 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:46.292768 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:46.293655 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 193227us
W20260709 13:20:46.302310 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 119014us
I20260709 13:20:46.303189 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:46.305119 16561 leader_election.cc:400] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:20:46.305424 16561 leader_election.cc:403] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:20:46.275941 23547 raft_consensus.cc:2764] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:20:46.273236 23483 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:46.308830 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:46.310720 17376 leader_election.cc:400] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:46.313071 17376 leader_election.cc:403] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:20:46.284587 23530 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:46.283154 23516 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:46.327596 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:46.301309 23490 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:46.294737 23335 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.296653 23421 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:46.333910 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.275300 23511 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:20:46.322453 23517 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:20:46.351591 16636 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:46.332438 23528 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:46.359303 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:46.358230 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:46.321046 23323 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:20:46.341166 23495 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:46.351787 23335 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:46.373549 23335 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:46.373958 23335 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:46.378072 23040 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:46.365164 23494 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:46.400106 16662 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:20:46.362936 23572 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14.
I20260709 13:20:46.356319 23551 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:46.415112 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 110059us
I20260709 13:20:46.363175 23411 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.420964 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:46.423619 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.424768 23542 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:46.425539 23542 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:46.426061 23542 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:46.411468 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:46.430275 17410 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:46.362737 23483 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.430706 23490 raft_consensus.cc:697] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:46.449321 16590 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
W20260709 13:20:46.449204 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 96793.5us
W20260709 13:20:46.452139 23602 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:20:46.456920 23411 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:46.416496 23389 consensus_queue.cc:1048] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:20:46.412138 23573 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.459985 23541 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:46.460851 23573 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.461125 16669 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:46.463191 23551 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:46.427518 23494 raft_consensus.cc:697] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:46.464641 23494 consensus_queue.cc:237] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.440819 23368 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:20:46.475651 23593 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:20:46.477330 23594 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:20:46.477478 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.478448 16590 raft_consensus.cc:2368] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:46.437266 23531 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:46.410295 23516 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.445814 23542 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.479907 23531 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.485051 16595 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:20:46.487985 23510 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:46.489131 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.489913 16595 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:46.492919 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.495528 23483 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:46.496997 17419 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:46.502476 23335 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.503984 23335 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:46.508738 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.514160 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.526759 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.550654 17374 leader_election.cc:400] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17.
I20260709 13:20:46.555338 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:46.545073 23517 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:46.547196 23606 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:46.571393 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:46.572072 16666 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:46.584584 23569 consensus_meta.cc:287] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.605s	user 0.013s	sys 0.001s
I20260709 13:20:46.585095 23569 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.559777 23503 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:46.588541 17376 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:46.603662 16666 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
W20260709 13:20:46.605618 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:46.606379 16478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 45316.1us
I20260709 13:20:46.509413 23490 consensus_queue.cc:237] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.609618 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.610246 16590 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:46.557492 23448 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:20:46.621045 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:46.636371 16590 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:20:46.658738 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.581045 23510 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 13:20:46.659993 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 249260us
I20260709 13:20:46.662338 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:46.620231 23421 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:46.666379 16666 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:46.666630 16591 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:20:46.663684 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 75285.5us
I20260709 13:20:46.662036 23569 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.663301 23606 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.685807 23606 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.689774 23606 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:46.692242 23609 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:20:46.701267 16634 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:46.703644 23618 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:20:46.704239 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:46.705034 16671 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:46.617470 23610 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:20:46.706841 23501 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:20:46.723022 23514 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.004s
I20260709 13:20:46.670836 23620 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:46.723619 23514 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.724529 23620 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:46.726279 23514 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:46.725773 23620 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.680325 16588 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:20:46.657581 23604 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:20:46.731413 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:46.740026 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:46.764494 16738 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:20:46.762202 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59168.6us
I20260709 13:20:46.677433 23511 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.772045 16740 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 22: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:46.766597 23517 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:46.766937 23522 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.829572 17412 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:46.727780 23530 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:46.882783 16594 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:20:46.883862 16589 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:46.887199 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.888067 16594 raft_consensus.cc:2393] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:20:46.889222 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 99549.9us
I20260709 13:20:46.782982 23511 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.891268 16743 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:46.891968 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:46.894832 16737 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:46.781778 23618 raft_consensus.cc:697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:46.896458 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:46.891968 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:46.893818 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:46.900033 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.900620 16742 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:46.904428 16745 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:46.911332 23302 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:46.911717 16592 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:46.912005 23302 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:46.911360 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:46.921623 16593 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:46.923977 17419 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:20:46.929129 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:46.846603 23517 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.847859 23522 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.918416 23607 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:20:46.943902 17419 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:46.947793 23595 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:46.949528 23511 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:46.904345 23569 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.963336 23595 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:46.963716 23569 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.964457 23595 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.966250 17416 raft_consensus.cc:1275] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:46.967810 23511 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:46.952483 23302 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:46.968591 23511 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:46.977797 16596 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:46.994774 16671 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:46.941004 23530 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:47.002279 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 105829us
I20260709 13:20:46.976521 23572 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.003522 22537 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:20:47.015938 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.016660 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:47.016734 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.019294 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:46.932677 23615 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
I20260709 13:20:47.024120 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:46.997634 23577 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:20:47.058166 16664 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 13:20:47.000954 23004 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:20:47.004329 23528 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:20:47.065150 17419 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:47.060864 16664 pending_rounds.cc:85] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 13
I20260709 13:20:47.066335 16664 pending_rounds.cc:107] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:20:47.066707 16664 raft_consensus.cc:2889] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:47.068809 23580 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 13:20:47.069829 23489 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.012029 23572 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:47.074805 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.070406 23489 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.079360 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.064096 23511 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:47.080739 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.081601 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.090701 16591 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:47.090909 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.027159 23618 consensus_queue.cc:237] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:47.096379 16665 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:20:47.063549 23604 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:20:47.078047 23613 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:47.107149 23641 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:47.087225 23489 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.110846 23636 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:47.111845 23636 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.114477 23636 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:47.115109 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51897us
I20260709 13:20:47.119977 23544 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:47.129871 23335 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.625s	user 0.009s	sys 0.004s
I20260709 13:20:47.131904 23522 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.134673 23572 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:47.136111 23517 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:47.137621 23483 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 13:20:47.160086 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.168687 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.169464 17410 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:47.197292 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:47.201097 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:47.199770 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.203910 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:47.219267 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.222029 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:47.222376 16587 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:20:47.223593 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 228630us
I20260709 13:20:47.205607 23624 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:47.228847 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.207983 23335 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:47.211519 23636 raft_consensus.cc:2764] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:20:47.211040 23642 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:47.253409 16743 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:20:47.221107 23528 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:20:47.248904 23547 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:47.270273 23623 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:20:47.264780 23547 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.285369 23547 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:47.280936 23642 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:20:47.299903 23646 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.344609 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.356220 16593 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:20:47.344615 23650 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.059s	user 0.002s	sys 0.002s
I20260709 13:20:47.358840 23569 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.352248 23646 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:20:47.365607 23657 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:20:47.372748 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:47.370522 23651 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 13:20:47.374110 23572 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:47.383176 16745 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2707) took 1268 ms. Trace:
I20260709 13:20:47.385874 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.387007 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.383687 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.389163 16593 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:20:47.383639 16745 rpcz_store.cc:276] 0709 13:20:46.115111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.115668 (+   557us) service_pool.cc:224] Handling call
0709 13:20:46.153973 (+ 38305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:46.911136 (+757163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.382940 (+471804us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b5400306954. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:47.383091 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725359360}
I20260709 13:20:47.374037 23646 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:47.399034 23633 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:20:47.416555 17419 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:20:47.443106 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.446455 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.445266 23577 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:47.475436 23635 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:47.430450 23425 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 13:20:47.507308 23637 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:47.499089 23547 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.504347 23577 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.444489 23606 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:20:47.514513 23606 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:47.514706 23577 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.515372 23547 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.517356 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.504534 23483 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.519412 23483 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.522078 23483 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.523030 16593 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:20:47.506282 23660 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:20:47.515022 23606 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:47.536371 23577 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:47.540066 23577 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:47.544488 23606 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:47.545778 23652 consensus_queue.cc:1048] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:47.549384 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.549554 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.542701 23577 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:47.551843 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.553108 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:47.558446 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:47.559145 16737 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:47.561574 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.559365 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.564551 23577 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:47.564774 23652 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:47.564611 23667 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:47.570226 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.572554 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:47.575235 23514 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:47.576287 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.576867 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:47.577128 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.577664 17420 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:20:47.578857 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.576895 16739 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:47.581017 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:47.581559 17411 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:47.583243 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.579547 23646 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:47.585153 23646 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.585258 23528 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:47.587414 23646 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:47.580842 23421 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:20:47.582427 23323 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:47.614221 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 114444us
I20260709 13:20:47.617105 16743 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:47.619951 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:47.620636 16669 raft_consensus.cc:606] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.623289 17411 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:47.634353 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.642719 16737 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:47.644233 16590 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:47.628450 23448 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:47.625394 23656 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:47.639168 23620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.016s	sys 0.004s
I20260709 13:20:47.658421 23299 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:47.678619 23299 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:47.686299 16587 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:20:47.688102 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:47.633494 23368 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:20:47.686512 23620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.020s	sys 0.020s
I20260709 13:20:47.703938 23299 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:47.705317 23299 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:47.707504 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:47.686183 23656 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.714169 16664 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:47.714890 16664 raft_consensus.cc:606] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:47.717281 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:47.772575 16736 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:20:47.708899 23448 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:47.715643 23517 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:47.796077 23604 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.796657 23604 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:47.783802 23523 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:47.799422 16594 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:20:47.738224 23667 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:20:47.784461 23448 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.788844 23517 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:47.808193 23666 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:20:47.812083 23604 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:47.794014 23572 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:47.817235 23572 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:47.812366 23675 log.cc:927] Time spent T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:20:47.804224 23523 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:47.830777 23662 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:20:47.806445 23517 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:47.834774 23656 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:47.837309 23663 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:20:47.839732 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.840423 16593 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:47.840684 23572 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:47.849035 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:47.806675 23448 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:47.877394 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.878907 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.856009 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:47.860561 23523 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:47.863262 23674 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:20:47.869874 23514 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:47.885641 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:47.886085 17413 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:20:47.885660 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.891054 16590 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:47.892436 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.895287 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:47.896135 16590 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:20:47.900344 23514 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:47.938334 23514 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:47.952220 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 61328.3us
W20260709 13:20:47.954320 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60742.4us
W20260709 13:20:47.955840 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:47.957605 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:47.882634 23604 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:48.069718 16664 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 20: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:47.862547 23517 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:48.138314 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 173704us
I20260709 13:20:47.954167 23572 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:47.989601 23514 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.056012 23604 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:47.965279 23577 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:48.154855 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:48.188611 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:48.163586 23604 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:48.171064 23623 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.158380 23572 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:48.190624 23577 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.234117 23577 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:48.235509 23676 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.349s	user 0.001s	sys 0.000s
I20260709 13:20:48.238504 23606 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:48.163178 23514 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.229163 22873 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.282663 23690 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.283311 23690 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:48.285403 23690 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:48.295778 23688 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:20:48.298161 23572 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:48.226500 23623 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.303285 23623 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:48.305370 23514 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.306457 22873 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.308830 22873 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.309645 16590 raft_consensus.cc:1275] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 13:20:48.269948 23299 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.000s
W20260709 13:20:48.298180 23649 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 1.033s	user 0.009s	sys 0.000s
I20260709 13:20:48.314639 23448 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.349870 23368 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:48.277102 23628 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:48.351567 23628 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.353858 23628 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.356086 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:48.329631 23547 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.357373 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:48.357717 23547 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.359225 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3691) took 1345 ms. Trace:
I20260709 13:20:48.360496 23547 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.359628 16663 rpcz_store.cc:276] 0709 13:20:47.014212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.014373 (+   161us) service_pool.cc:224] Handling call
0709 13:20:47.014804 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.015271 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.015309 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.035174 (+ 19865us) log.cc:844] Serialized 26 byte log entry
0709 13:20:47.192897 (+157723us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:47.192926 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:47.192973 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.297617 (+1104644us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:48.358916 (+ 61299us) raft_consensus.cc:1685] finished
0709 13:20:48.358939 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.359165 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2468,"raft.queue_time_us":8078,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":32819,"spinlock_wait_cycles":2536832,"thread_start_us":111117,"threads_started":1,"wal-append.queue_time_us":194337}
I20260709 13:20:48.360666 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:48.335079 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:48.365712 23368 raft_consensus.cc:993] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:48.367254 23618 raft_consensus.cc:1081] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:48.357079 16671 raft_consensus.cc:606] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:48.369148 16671 raft_consensus.cc:614] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:48.370029 23636 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:48.350579 23448 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.373550 23623 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:48.375092 23634 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:48.376276 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:48.375888 23634 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:48.376958 16664 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:48.382228 23517 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.403940 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:48.405005 16671 raft_consensus.cc:606] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:48.405524 16671 raft_consensus.cc:614] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:48.434836 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 219151us
I20260709 13:20:48.448588 16671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee99829f455541f09dac21015b00282e"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:48.449584 16671 raft_consensus.cc:606] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:48.450080 16671 raft_consensus.cc:614] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:20:48.468801 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 328696us
I20260709 13:20:48.491442 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:48.498188 16593 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.010s	sys 0.008s
W20260709 13:20:48.500218 16593 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.010s	sys 0.008s
W20260709 13:20:48.499306 17416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.007s	sys 0.010s
W20260709 13:20:48.501888 17416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.007s	sys 0.011s
I20260709 13:20:48.502383 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:48.505631 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:48.506404 16737 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:20:48.508874 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.491482 23634 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:48.513803 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:48.514652 16740 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:20:48.524936 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:48.525790 16741 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:48.508025 23542 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:48.515803 23448 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.571249 23634 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:48.568171 23687 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.365s	user 0.003s	sys 0.000s
W20260709 13:20:48.582753 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 224142us
I20260709 13:20:48.588989 16740 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
W20260709 13:20:48.585832 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 150062us
I20260709 13:20:48.565681 23604 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:48.608558 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 243318us
I20260709 13:20:48.606313 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:48.583750 23615 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:48.612457 16737 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:48.652205 16736 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:20:48.583660 23448 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.584609 23634 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.672015 16591 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:20:48.672768 16591 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:48.673166 16591 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:48.648175 23040 consensus_queue.cc:1048] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 13:20:48.683264 16589 raft_consensus.cc:1275] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:20:48.649761 23615 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:48.733633 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 519434us
I20260709 13:20:48.728936 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:48.735038 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:48.742369 16591 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.769120 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:48.769937 17410 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:48.728674 23692 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.789358 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:48.790625 17410 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:48.728880 23694 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:48.797137 22537 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:48.798218 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.803030 22537 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:48.806077 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:48.736644 23621 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:48.749784 23623 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:48.744419 23517 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.808465 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:48.811036 16591 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.812012 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:48.812467 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 77515.8us
I20260709 13:20:48.823781 23720 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.771642 23628 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.878988 22537 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:48.881917 17410 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:20:48.893602 17376 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:48.830701 23646 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:48.899595 23610 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:48.808984 23522 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
W20260709 13:20:48.913321 23695 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:20:48.912897 23610 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:48.915467 23700 log.cc:927] Time spent T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:20:48.829799 23621 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:48.916253 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:48.917075 16595 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:48.917349 23621 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:48.877364 23623 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.918256 23623 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:48.919561 23697 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:20:48.920179 23634 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.920614 23623 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:48.915509 23714 log.cc:927] Time spent T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:20:48.921206 23623 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:48.921399 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1480) took 1856 ms. Trace:
I20260709 13:20:48.921859 23623 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.830351 23517 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.807755 23696 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.117s
I20260709 13:20:48.846570 23550 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:48.810379 23692 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.924445 23517 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.922312 23703 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.861716 23699 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:48.922012 17416 rpcz_store.cc:276] 0709 13:20:47.064701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.064823 (+   122us) service_pool.cc:224] Handling call
0709 13:20:47.065106 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:47.711069 (+645963us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b1800045ee0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327f78c4bc 00007f327cada48b 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf4455 00007f327fbdc060 00007f327fbdcbfb 00007f327fbe764b
0709 13:20:47.711192 (+   123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:47.711216 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:47.711773 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:20:48.504970 (+793197us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:48.505012 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.505084 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:48.920965 (+415881us) raft_consensus.cc:1685] finished
0709 13:20:48.920996 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:48.921292 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24125,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":5223,"spinlock_wait_cycles":34511744,"thread_start_us":827864,"threads_started":2,"wal-append.queue_time_us":943289}
I20260709 13:20:48.924090 23604 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:48.955552 16745 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 19. Ops: []
I20260709 13:20:48.942535 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:48.957499 16739 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:48.814095 23694 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.936131 23517 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:48.880362 23628 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.954694 23577 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:48.901746 23605 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:20:48.858999 23720 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.925796 23550 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:48.972301 23448 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:48.958806 23699 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:48.957669 23703 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:49.014801 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:48.995915 23692 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.024044 22537 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:48.980587 23517 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.024634 22537 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.003811 23707 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:48.993309 23577 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.044687 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:49.045612 16664 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.022857 23610 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.049767 22537 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.049899 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.050308 22537 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:49.051789 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 152260us
I20260709 13:20:49.051992 23724 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.052572 23724 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.047675 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:49.055130 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 282560us
I20260709 13:20:49.056567 23319 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.057181 23319 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.050900 22537 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.035749 23692 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.066392 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:49.076550 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:49.076998 16666 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:49.077452 16666 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:49.068436 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.068862 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.074661 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.078549 16662 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:20:49.069173 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.069440 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.066785 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.080905 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.088973 22537 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.089792 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.090229 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.091117 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:49.092103 23708 log.cc:927] Time spent T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:20:49.086524 16666 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.092983 23694 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.065992 23724 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.093293 16666 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:49.088971 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:49.094528 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:49.071820 23701 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 13:20:49.064774 23710 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:20:49.067649 23550 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.070825 23610 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.063537 23722 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.037781 23707 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:49.114432 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58175.7us
I20260709 13:20:49.118840 23577 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.119256 16593 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4822) took 1276 ms. Trace:
W20260709 13:20:49.120520 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 67346.3us
I20260709 13:20:49.121420 17376 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:49.119684 16593 rpcz_store.cc:276] 0709 13:20:47.848044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.849121 (+  1077us) service_pool.cc:224] Handling call
0709 13:20:47.915902 (+ 66781us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:48.500600 (+584698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:48.500633 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:48.501130 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 13:20:48.660801 (+159671us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:48.660838 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.660909 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:49.118980 (+458071us) raft_consensus.cc:1685] finished
0709 13:20:49.119002 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.119160 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38979,"raft.queue_time_us":285666,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":11004,"thread_start_us":618778,"threads_started":2,"wal-append.queue_time_us":231541}
I20260709 13:20:49.122560 17413 raft_consensus.cc:1275] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:20:49.124193 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.064716 23728 consensus_queue.cc:1048] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:20:49.126732 16666 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:49.130545 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.108464 23604 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:49.132997 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.141961 16664 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:49.163378 22537 raft_consensus.cc:993] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:49.128033 23718 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:49.165948 23718 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.132036 23724 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.166414 23718 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.166333 23522 raft_consensus.cc:1081] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:20:49.169301 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:20:49.173411 23719 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:20:49.179268 23623 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.180881 23703 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.164551 23605 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:49.181010 23684 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.877s	user 0.004s	sys 0.000s
I20260709 13:20:49.192279 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.115145 23610 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.195242 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.198288 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.211496 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:49.121304 23722 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.229385 23722 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.230358 16667 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:49.231536 23299 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.232260 23299 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.232744 23610 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.218561 23646 consensus_queue.cc:1048] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:20:49.214602 23726 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:49.237035 23726 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.203656 23605 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:20:49.239328 23605 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.208046 23718 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.239996 23605 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:49.219798 23699 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.222653 23720 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.223251 23730 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.225119 23517 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.185195 23628 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.226943 23704 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.239449 23726 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.221464 23707 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.166523 23724 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.251775 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:49.253154 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 124633us
I20260709 13:20:49.254628 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.234776 23299 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:49.260164 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.260974 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:49.264137 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:49.258347 23628 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.259057 23726 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.252849 23730 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.271783 23522 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:20:49.295674 23732 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:20:49.297185 23730 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.299273 23718 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.278218 23490 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:49.300742 23693 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.301446 23483 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:49.301338 23693 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.302078 23483 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.304114 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.304777 16663 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:49.305472 23698 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:49.258724 23704 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.306118 23698 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.307561 23704 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.308429 23698 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.308951 23698 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.309429 23698 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.310835 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:49.311347 23517 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.311779 16593 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:20:49.311875 23517 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.313145 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:49.314114 23517 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.314381 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:49.315569 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:49.316195 16591 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:49.319480 16595 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16. Ops: []
W20260709 13:20:49.321162 23734 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:49.321794 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:49.322921 16593 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:49.305565 23483 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.277031 23628 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.326275 23628 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.302975 23694 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:49.278889 23724 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:49.328416 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.352662 23698 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.327589 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:49.354087 16738 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.355439 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:49.348505 23693 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.351188 23726 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.348914 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:49.357445 23698 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.364799 16559 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:49.365549 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:49.371354 16738 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:49.371945 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.359606 23628 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:49.370353 23489 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:49.376345 23489 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.377892 23489 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:49.379377 23525 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:49.379966 23525 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.383877 23525 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.369309 23696 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:49.369899 23693 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:49.387573 23605 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.389478 23319 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.388295 23693 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:49.390566 23693 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.393409 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.395006 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.392906 23605 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.375928 23628 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:49.398835 23628 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:49.399775 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.407289 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1976) took 1091 ms. Trace:
W20260709 13:20:49.409644 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 146840us
I20260709 13:20:49.410687 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:49.407680 17411 rpcz_store.cc:276] 0709 13:20:48.316286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.317000 (+   714us) service_pool.cc:224] Handling call
0709 13:20:48.356437 (+ 39437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.356703 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:20:48.585286 (+228583us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.237522 (+652236us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b1800396160. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:20:49.402582 (+165060us) spinlock_profiling.cc:243] Waited 8.34 ms on lock 0x7b1800396160. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327f78c4bc 00007f327cada48b 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287
0709 13:20:49.402743 (+   161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:49.402822 (+    79us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:49.407047 (+  4225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.407126 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.407231 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36687,"prepare.queue_time_us":34766,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":133959,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":15512,"spinlock_wait_cycles":83153920,"thread_start_us":475194,"threads_started":3,"wal-append.queue_time_us":239566}
I20260709 13:20:49.410566 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:49.361272 23694 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.416910 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:49.369002 23698 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.418043 23319 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:49.437446 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.443471 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.438143 16669 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:49.414448 23696 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:49.426460 23595 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:49.458443 23605 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:49.463291 23605 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.465792 23605 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.476718 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.477455 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:49.480273 23694 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.484055 23698 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.486840 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.432160 23737 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:49.451357 23490 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.452750 23517 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.487244 23529 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.515992 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:49.526706 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 166746us
I20260709 13:20:49.497887 23615 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.513496 23621 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.503134 23489 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.523015 23740 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:49.528184 23542 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.527953 23529 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.495329 23517 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.495131 23490 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.630155 23726 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.631038 23490 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.632599 23628 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.633543 23517 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.639539 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:49.616428 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85022.5us
I20260709 13:20:49.623018 23580 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:49.622941 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.627372 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.663941 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.664831 16662 raft_consensus.cc:2410] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:20:49.664999 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.666239 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:49.614782 23615 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.614248 23740 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.671504 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.672792 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.674651 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.682611 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.684589 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.668977 23580 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.676144 23740 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:49.689630 23580 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.690850 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.692181 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.693389 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:49.694100 16662 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:49.543115 23621 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.707114 23621 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.535446 23696 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.680821 23735 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:49.713377 23735 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:49.713853 23735 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:49.681061 23615 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.721953 23615 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.620136 23542 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.736347 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.739032 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.739943 16738 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:49.741204 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.742784 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.743660 16741 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:49.746675 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.747807 16737 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:49.749265 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.751132 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.776108 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.778461 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.795218 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.796168 16738 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:49.797833 16740 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:49.919628 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4364) took 1633 ms. Trace:
I20260709 13:20:49.923132 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:49.926527 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.933297 16671 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:49.920454 16736 rpcz_store.cc:276] 0709 13:20:48.286439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.286593 (+   154us) service_pool.cc:224] Handling call
0709 13:20:48.287043 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:48.652908 (+365865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.919188 (+1266280us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540005ad54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:49.919526 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838940032}
I20260709 13:20:49.941425 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.718425 23529 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:49.946583 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1355) took 2134 ms. Trace:
I20260709 13:20:49.947649 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.948715 17410 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:49.951809 23737 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.952441 23737 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.953140 23638 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:49.947768 16592 rpcz_store.cc:276] 0709 13:20:47.792283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.792417 (+   134us) service_pool.cc:224] Handling call
0709 13:20:48.315933 (+523516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.316186 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:48.316223 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:48.316284 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:20:48.316339 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:48.316423 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:49.926510 (+1610087us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54001fb3d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:49.926698 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195807104}
I20260709 13:20:49.954380 23605 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.954888 23738 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.953714 23638 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.954764 23646 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.956521 23646 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.957633 23638 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.957759 23490 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:49.958503 23490 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:49.960605 23490 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.958665 23646 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:49.955497 23738 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.954746 23737 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:49.961248 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 166675us
W20260709 13:20:49.964169 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:49.962679 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.971720 16736 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:20:49.972321 16736 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:49.767935 23542 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:49.985387 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.977777 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.986738 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:49.984197 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:49.978030 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:49.775068 23726 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:49.975597 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:50.021502 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45971.2us
I20260709 13:20:49.972875 16736 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.026161 16662 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:49.946331 23693 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.125849 23738 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.159228 23640 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:50.160269 23640 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.047597 23628 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.162477 23640 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.018967 23730 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.145s
I20260709 13:20:50.164004 23638 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.049086 23726 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.164635 23638 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.073258 16736 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.167258 23638 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.129756 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.169377 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:50.169945 16671 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:50.170218 16671 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:50.170855 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.171643 16743 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:50.171195 16671 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.027882 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.172487 16671 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.174448 16736 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.176244 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1978) took 1070 ms. Trace:
I20260709 13:20:50.176625 17416 rpcz_store.cc:276] 0709 13:20:49.106210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.106376 (+   166us) service_pool.cc:224] Handling call
0709 13:20:49.106725 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.128690 (+ 21965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:49.128721 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:49.128766 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:49.128826 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.176032 (+1047206us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b54005d25d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:50.176077 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.176190 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50062208}
I20260709 13:20:50.166527 23726 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.177966 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:50.178390 23638 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.178486 17416 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.182919 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4832) took 2017 ms. Trace:
I20260709 13:20:50.184629 16596 rpcz_store.cc:276] 0709 13:20:48.165263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.237734 (+ 72471us) service_pool.cc:224] Handling call
0709 13:20:48.237970 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:48.279368 (+ 41398us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:20:48.496137 (+216769us) log.cc:844] Serialized 14 byte log entry
0709 13:20:48.683976 (+187839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:48.684043 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:48.684125 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.972035 (+1287910us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b1800380ce0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419
0709 13:20:50.182679 (+210644us) spinlock_profiling.cc:243] Waited 41.1 ms on lock 0x7b540010f754. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:50.182740 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:50.182843 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29922,"prepare.queue_time_us":486152,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":9830,"raft.queue_time_us":497,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":15034,"spinlock_wait_cycles":103710720,"thread_start_us":383016,"threads_started":2,"wal-append.queue_time_us":512839,"wal-append.run_cpu_time_us":1899,"wal-append.run_wall_time_us":1102997}
I20260709 13:20:50.187397 23638 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.197396 23638 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.167630 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:50.167294 17413 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.079s	sys 0.188s
W20260709 13:20:50.200169 17413 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.080s	sys 0.189s
W20260709 13:20:50.180377 17414 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.023s	sys 0.054s
W20260709 13:20:50.204121 17414 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.024s	sys 0.054s
I20260709 13:20:50.210655 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.227264 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.228026 17411 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:50.228761 23448 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.229296 23448 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.231402 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3702) took 1161 ms. Trace:
I20260709 13:20:50.231812 16667 rpcz_store.cc:276] 0709 13:20:49.069765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.090562 (+ 20797us) service_pool.cc:224] Handling call
0709 13:20:49.090824 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:49.230811 (+139987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:50.231174 (+1000363us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54000f37d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:50.231340 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319839488}
I20260709 13:20:50.233518 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.234258 23580 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:20:50.237485 23595 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.000s
I20260709 13:20:50.238026 23528 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.238579 23528 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.240839 23448 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.241415 23448 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.241847 23448 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.246774 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.249789 17416 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:50.249828 17415 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
W20260709 13:20:50.232719 23735 consensus_meta.cc:287] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.003s
I20260709 13:20:49.971624 23707 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:50.200368 23517 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:50.295943 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.297503 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.298695 16669 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:20:50.296952 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.120488 23389 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:50.302227 23389 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.304491 23389 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.314625 23517 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.317078 23517 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.317639 23517 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.318167 23517 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:49.972731 23605 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.320866 23605 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.323520 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.326121 16671 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:50.329111 16591 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:50.018994 23742 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.038763 23646 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:50.327499 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:50.331233 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:50.038321 23490 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:50.337248 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:50.342016 16745 raft_consensus.cc:2393] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:50.344256 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.355002 23754 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.355821 23754 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.296082 23735 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.365119 16663 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:20:50.366353 23735 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:50.371730 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 44312.2us
I20260709 13:20:50.333313 23646 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:50.386211 16591 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:50.358635 23754 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.345304 23531 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:50.265220 23693 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.344471 23643 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.324229 23707 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.398448 16591 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:20:50.383620 23299 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:50.406543 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.416945 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.391640 23646 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.390259 23528 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.349833 23490 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.420073 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.346010 23742 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.423913 16593 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:50.427521 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.424978 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.428112 16593 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:50.428530 16587 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:50.429409 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.393708 23531 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.425770 23448 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.431329 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.433131 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:50.434046 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.435166 16587 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:20:50.443789 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 101076us
I20260709 13:20:50.445112 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.423960 23299 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:20:50.450484 23752 log.cc:927] Time spent T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:20:50.450584 23299 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.398080 23643 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.445778 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.445674 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:50.455291 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.433931 23531 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:50.479955 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.453867 16634 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:50.487916 23763 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:50.488510 23763 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:50.489499 23763 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.493091 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.494132 16588 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:50.481429 16593 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:20:50.517518 16588 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:50.519686 23693 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.548127 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.553100 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.554198 16587 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:50.556421 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.557265 16588 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
W20260709 13:20:50.571043 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 69597.9us
W20260709 13:20:50.560875 23769 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:20:50.572821 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.573748 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:50.585564 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:50.591508 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.592238 16588 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:20:50.597638 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.589425 23726 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.614482 22537 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:50.615100 22537 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:50.615691 22537 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:50.606805 23636 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:50.617193 23778 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:50.617270 23636 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:50.617906 23636 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:50.623458 23643 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.636353 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.637153 16596 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:50.659957 22537 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.613925 23742 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.696493 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.697849 16596 raft_consensus.cc:2410] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:20:50.700937 22537 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:50.703204 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 105640us
I20260709 13:20:50.639420 23707 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.645010 23517 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:50.704185 23762 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:20:50.617877 23778 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.630824 23726 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:50.745674 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 144770us
I20260709 13:20:50.691197 23692 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.703881 23389 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:50.758792 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.716753 23490 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.706305 23723 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:50.754823 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:50.707055 23510 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:50.783859 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79157.3us
I20260709 13:20:50.739168 23707 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:50.799650 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.743880 23636 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.819196 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.783766 23692 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.793686 23510 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.792989 23723 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.839496 23723 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.841006 23489 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.843905 23778 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.819031 23763 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:50.844913 23724 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:20:50.845330 23724 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:50.845925 23763 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:50.846165 23724 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.847036 23763 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.847788 23724 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:20:50.834715 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:50.852316 23489 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:50.856084 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:50.829262 23692 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.829023 23646 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.831362 23510 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.832170 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.833982 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.834684 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.797197 23781 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:20:50.835305 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.836181 23636 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:50.836462 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.871205 16592 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:50.800913 23707 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.836210 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:50.857137 17415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48636 (request call id 1082) took 3294 ms (client timeout 1893 ms). Trace:
W20260709 13:20:50.858598 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 103862us
W20260709 13:20:50.827375 23776 log.cc:927] Time spent T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:20:50.874655 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.859381 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:50.876752 23742 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:50.877682 23531 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.860737 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:50.878754 16662 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:50.861959 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.873932 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.867796 16671 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:50.870114 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.878620 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:50.870119 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.871447 16596 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:50.871483 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.883404 16593 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
I20260709 13:20:50.873277 16587 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 14.
W20260709 13:20:50.873219 17415 rpcz_store.cc:269] 0709 13:20:47.563088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.563206 (+   118us) service_pool.cc:224] Handling call
0709 13:20:50.856942 (+3293736us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b54005802d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:50.857040 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6141329152}
W20260709 13:20:50.868788 23740 consensus_meta.cc:287] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.179s	user 0.014s	sys 0.000s
I20260709 13:20:50.860113 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.886471 16591 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:20:50.887809 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.888857 16595 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:50.890298 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.888489 16589 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:50.890925 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.867883 23489 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:50.892756 23643 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.893426 23618 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:50.887197 23742 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:50.895130 16662 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:50.920920 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.921911 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.923012 16671 raft_consensus.cc:2393] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:50.897802 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.898495 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:50.900218 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.901737 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:50.923930 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.924368 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.904742 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:50.905457 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:50.906670 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.925988 16737 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:50.910476 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:50.924398 16742 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:50.918213 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.928892 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.930126 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:50.931152 16671 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:50.933203 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:50.936468 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.942538 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:50.938060 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.921286 23618 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:50.940177 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.938558 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.950253 16592 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:50.951984 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.921425 23503 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:50.931839 23510 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:50.949802 23490 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.955139 23618 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.956260 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.956689 23618 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:50.955302 23490 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:50.958341 16708 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.881s (SENT) [suppressed 2 similar messages]
I20260709 13:20:50.958819 23490 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:50.958971 23727 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:50.944103 23368 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:50.959739 23707 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:50.956972 17415 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:50.960209 23784 raft_consensus.cc:2764] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:20:50.921548 23636 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.961161 16710 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:50.944262 23742 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:50.962692 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.967226 17415 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:50.962705 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.965358 23636 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:50.889997 23740 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:50.950209 23726 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:50.969780 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:50.937860 23789 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:50.970096 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.941241 23786 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:50.973364 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.963244 23727 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:50.983719 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.973577 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.973577 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.975879 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.985324 16741 raft_consensus.cc:2368] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:50.976919 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.979580 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.986399 16736 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:50.987751 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:50.988564 17411 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:50.969928 23577 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:50.990123 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.991760 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:50.993040 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:50.994025 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:50.994709 16736 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 20: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:50.992667 23577 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.002697 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.007081 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:51.004204 23531 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.010811 23577 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:20:51.017663 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59344.5us
I20260709 13:20:51.029583 16737 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 10. Current term is 11. Ops: []
I20260709 13:20:51.029770 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:51.030706 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.034891 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:51.041513 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:51.044157 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:51.045789 23757 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:51.046707 23758 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:51.047339 23758 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:51.001267 23628 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:51.052568 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.053480 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:51.049289 16736 raft_consensus.cc:2393] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 19: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:51.050776 17411 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:20:51.055307 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:50.988919 23786 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.058779 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:51.062116 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:51.061316 16708 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.047371 23757 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:20:51.065630 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:51.065696 16662 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:51.069542 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.068336 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.066577 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.060866 23646 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.077451 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:51.079352 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 146214us
I20260709 13:20:51.083096 16738 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:51.083887 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.084296 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.088758 23786 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.089612 23757 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:20:51.091688 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.095369 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.096042 17414 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:51.101136 23758 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.103493 23503 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:51.104926 23503 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:51.105420 23503 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:51.070626 23531 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:51.081674 16561 connection.cc:256] RPC call timeout handler was delayed by -1.7438912049999544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:20:51.119575 16561 connection.cc:256] RPC call timeout handler was delayed by -1.639491205000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18930000000000002s for a response
W20260709 13:20:51.120211 16561 connection.cc:256] RPC call timeout handler was delayed by -1.4153247700000975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1799s for a response
W20260709 13:20:51.120715 16561 connection.cc:256] RPC call timeout handler was delayed by -1.0072576290001507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
I20260709 13:20:51.116974 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.125072 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.126114 23726 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.126636 23726 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:51.127084 23726 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.128417 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.129914 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.078316 23490 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:51.064636 23530 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:20:51.132880 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.134258 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.136924 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.139171 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.123852 23646 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:51.140738 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.111197 23636 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.166339 16736 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
W20260709 13:20:51.165604 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46622.7us
I20260709 13:20:51.142660 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.170660 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.106573 23503 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.064457 23577 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.179844 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:20:51.203881 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 126722us
I20260709 13:20:51.065228 23742 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.205086 17412 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:51.146445 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.141283 23604 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:51.214015 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.150619 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:51.216172 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 66766.6us
W20260709 13:20:51.209409 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 84778.5us
I20260709 13:20:51.217146 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.133917 23490 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:51.165198 23605 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.219501 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.141635 23737 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:51.220232 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73930.2us
I20260709 13:20:51.227006 23715 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:51.227551 23738 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.240809 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.141348 23693 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:51.076151 23727 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.215987 23503 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:51.275180 23605 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:51.277917 23758 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:51.273011 23693 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:51.287884 23693 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.288910 23693 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:51.289671 23577 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.230934 23604 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.295332 23789 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:51.297405 23604 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.270918 23740 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:51.275708 16708 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.306041 17412 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
W20260709 13:20:51.306058 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 125455us
W20260709 13:20:51.276682 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57259.4us
I20260709 13:20:51.218742 23490 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.310882 17374 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:51.329268 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53651.6us
I20260709 13:20:51.330399 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.332001 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.337836 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:20:51.339335 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 122273us
I20260709 13:20:51.339627 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.343255 23531 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:51.345307 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.318821 23715 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:51.347585 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.354949 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:51.328349 23741 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:51.342281 23577 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:51.369000 23577 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.369480 23577 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.371183 16594 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4848) took 2625 ms. Trace:
I20260709 13:20:51.371624 16594 rpcz_store.cc:276] 0709 13:20:48.745953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:48.746116 (+   163us) service_pool.cc:224] Handling call
0709 13:20:48.747913 (+  1797us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:20:48.748319 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:48.748358 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:20:48.749104 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 13:20:48.830845 (+ 81741us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:49.008441 (+177596us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.009763 (+  1322us) log.cc:844] Serialized 14 byte log entry
0709 13:20:49.298142 (+288379us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b18000ec8c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f327fbdc060 00007f327fbee716 00007f327fbff287 00007f327fbff208
0709 13:20:49.298336 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:49.298416 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:50.131721 (+833305us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b18000ec8c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1
0709 13:20:50.228530 (+ 96809us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b18000ec8c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf7522 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:20:51.367586 (+1139056us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b18000ec8c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:20:51.370955 (+  3369us) raft_consensus.cc:1685] finished
0709 13:20:51.370978 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.371123 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14578,"prepare.queue_time_us":45737,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":4362,"raft.queue_time_us":298064,"raft.run_cpu_time_us":465,"raft.run_wall_time_us":50334,"spinlock_wait_cycles":34779392,"thread_start_us":191758,"threads_started":2,"wal-append.queue_time_us":233709}
I20260709 13:20:51.376838 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:51.348421 23738 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.382485 23738 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.360319 23531 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:51.346645 23786 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:51.339730 23646 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:20:51.387216 23646 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:51.388304 23646 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.335863 23530 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:51.382937 23738 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.358922 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:51.394973 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44729.3us
W20260709 13:20:51.402794 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.396319 16592 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:51.417232 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.363669 23794 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:20:51.423520 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 206442us
I20260709 13:20:51.424355 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:51.427946 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 69157.7us
I20260709 13:20:51.429163 23740 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:51.429651 23740 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.429922 23728 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:51.430078 23740 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:51.438176 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.439095 23529 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:51.439487 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.440383 23529 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.440542 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.391084 23741 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.443987 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.444626 16664 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
W20260709 13:20:51.445964 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52532us
I20260709 13:20:51.446000 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:51.446899 16667 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:51.457975 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.461331 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.463325 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.467769 23531 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.468362 23531 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.470345 23531 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:51.460677 23757 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.493193 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3727) took 1256 ms. Trace:
I20260709 13:20:51.493839 16663 rpcz_store.cc:276] 0709 13:20:50.237123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.252719 (+ 15596us) service_pool.cc:224] Handling call
0709 13:20:50.287454 (+ 34735us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:50.365557 (+ 78103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.492833 (+1127276us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b54000f37d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:51.493105 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178008448}
I20260709 13:20:51.499795 23781 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:51.501559 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.503888 16667 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:51.508205 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.516088 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.521855 16636 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.531255 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.535852 16636 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.530277 23368 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.537524 23495 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.539278 23495 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:51.539709 23495 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.517282 23595 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:51.517802 23742 raft_consensus.cc:2764] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:20:51.544000 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:51.521963 23796 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:51.553668 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.554337 17409 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:51.515954 23803 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:51.525159 23727 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:51.556339 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.563364 17409 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
W20260709 13:20:51.564354 16741 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 4379) took 1818 ms (client timeout 1973 ms). Trace:
W20260709 13:20:51.564889 16741 rpcz_store.cc:269] 0709 13:20:49.745588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:49.748895 (+  3307us) service_pool.cc:224] Handling call
0709 13:20:51.564074 (+1815179us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400426154. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:51.564252 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2840497536}
I20260709 13:20:51.567513 16492 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.569154 16492 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.571211 16492 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 19 to 23, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 23 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.574121 16492 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.586743 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.581151 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:51.588057 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.550979 23628 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:51.555830 23794 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:20:51.589133 23741 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.601022 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:51.602631 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:51.612010 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:51.612859 17413 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:51.562112 23796 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:51.619406 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.597546 23643 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:51.591836 23628 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:51.613874 23779 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.827s	user 0.004s	sys 0.012s
I20260709 13:20:51.599171 23737 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:51.609438 23638 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.632592 17413 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:51.620471 23490 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.644876 23299 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.673759 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 1995) took 1683 ms. Trace:
I20260709 13:20:51.666142 17412 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
W20260709 13:20:51.676349 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44151.4us
I20260709 13:20:51.629231 23628 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:51.633922 23737 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:51.674420 17410 rpcz_store.cc:276] 0709 13:20:49.989936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:50.169313 (+179377us) service_pool.cc:224] Handling call
0709 13:20:50.169562 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:50.424416 (+254854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:50.424446 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:50.424502 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:50.424532 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:51.673344 (+1248812us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b540073b254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:51.673419 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:51.673676 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585695616}
I20260709 13:20:51.641031 23577 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.686000 23577 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:51.687011 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.687754 23577 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:51.689055 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.689723 23577 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:51.690210 23577 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:51.690654 23577 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:51.692472 23490 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.692725 16738 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:51.694129 23510 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:51.695163 23510 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:51.695515 23510 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:51.696352 23510 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.697641 23510 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:51.699959 23577 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.694669 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:51.702018 23490 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:51.702694 23490 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:51.702931 23737 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:51.703133 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:51.703538 23490 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.707598 23646 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:51.708598 23807 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:51.708478 23796 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.708993 23807 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:51.709766 23807 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.711007 23807 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:51.681514 23726 consensus_meta.cc:287] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 13:20:51.724433 23726 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.725961 16741 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:51.720925 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.722286 23740 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.681664 23299 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.726665 23726 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.642526 23495 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:51.747080 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 283877us
I20260709 13:20:51.751737 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:51.753697 23489 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:20:51.754134 23489 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:51.720327 23738 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.756853 17412 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
W20260709 13:20:51.757030 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 221319us
I20260709 13:20:51.759951 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:51.757560 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.763599 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.766249 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.762836 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:51.773912 16663 raft_consensus.cc:1240] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:51.774315 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.775018 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.775931 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.743340 23737 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.755231 23489 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.776710 23692 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:51.781533 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.782418 23495 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:51.784132 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:51.784646 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.786545 23806 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.787078 23806 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.788093 23740 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:51.789191 23806 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.769397 23757 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:51.792621 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:51.793284 23737 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.795423 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.797956 23510 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:51.799000 23510 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:51.799258 23299 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:51.799356 23510 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:51.800370 23510 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.808007 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:51.793342 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:51.814427 16743 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:51.778450 23808 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:20:51.815315 23808 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:51.815778 23808 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:51.818114 16741 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:51.819358 23692 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:51.820393 23489 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:51.795491 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:51.821508 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:51.828924 17412 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:20:51.824365 16743 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:51.826308 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:51.832513 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:51.832859 17411 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:51.837900 16741 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:51.840868 23738 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:51.850090 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:51.854918 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79987.9us
I20260709 13:20:51.855976 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.849423 17412 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:51.859382 23510 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.863353 16743 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:51.827411 23692 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.865067 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:51.866041 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.868113 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:51.869931 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.863227 23693 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:51.870970 23727 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:51.871778 23814 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:51.827162 23742 consensus_queue.cc:1059] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:20:51.879668 23742 raft_consensus.cc:3055] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:51.880079 23742 raft_consensus.cc:740] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:51.880869 23742 consensus_queue.cc:260] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:51.881995 23742 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:51.865729 23811 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:51.887002 23784 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:51.887461 23784 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:51.887799 23784 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:51.891374 16739 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:20:51.891796 16739 pending_rounds.cc:85] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 10
I20260709 13:20:51.892009 16739 pending_rounds.cc:107] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:20:51.892257 16739 raft_consensus.cc:2889] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:51.897240 23693 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:51.902647 16492 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 16 to 19, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.907452 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:51.911656 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 43596us
I20260709 13:20:51.911733 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:51.912853 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:51.914633 16663 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:20:51.915015 22537 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.916911 16589 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:20:51.920251 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:51.924211 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:51.925112 16738 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:51.927224 16739 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:51.927687 23693 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:51.911314 23723 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:51.935329 23577 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:51.941658 23628 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.922693 16710 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:51.946863 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.947324 17412 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:20:51.950832 23723 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:51.954640 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.956815 23807 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:51.947793 23335 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:51.957387 23807 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:51.957780 23335 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:51.958235 23335 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:51.959141 23577 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:51.959774 23577 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:51.961086 23811 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:51.960811 23577 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:51.958474 23807 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:51.972796 23816 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.115s	user 0.002s	sys 0.000s
I20260709 13:20:51.975704 16492 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 13 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 16 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.977682 16492 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 15 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:20:51.989719 22537 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:20:51.983199 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:51.992739 16710 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:51.999917 23646 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:52.000412 23646 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:52.000801 23646 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.000694 16710 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:52.002621 23299 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:52.003384 23299 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:52.004357 23299 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.005388 16491 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 306) took 8492 ms. Trace:
I20260709 13:20:52.006516 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:52.008260 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:52.012286 23784 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.014496 16739 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:52.020565 23335 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.022300 16666 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:52.063443 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:52.062716 16742 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:51.951643 23656 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:52.079186 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:20:52.083369 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 158087us
W20260709 13:20:52.097069 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.037985 16491 rpcz_store.cc:276] 0709 13:20:43.513475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:43.513952 (+   477us) service_pool.cc:224] Handling call
0709 13:20:43.610069 (+ 96117us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:20:52.005327 (+8395258us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:46.374681 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:46.375364 (+   683us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:46.375402 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:20:46.375413 (+    11us) tablet.cc:662] Decoding operations
0709 13:20:46.376354 (+   941us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:46.376412 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:46.376442 (+    30us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:20:46.377146 (+   704us) tablet.cc:701] Row locks acquired
0709 13:20:46.377166 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:20:46.377310 (+   144us) write_op.cc:270] Start()
0709 13:20:46.377421 (+   111us) write_op.cc:276] Timestamp: P: 1783603246377261 usec, L: 0
0709 13:20:46.377444 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:20:46.381771 (+  4327us) log.cc:844] Serialized 3181 byte log entry
0709 13:20:46.683051 (+301280us) op_driver.cc:464] REPLICATION: finished
0709 13:20:46.918443 (+235392us) write_op.cc:301] APPLY: starting
0709 13:20:46.918520 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:46.957185 (+ 38665us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:46.957322 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:47.127527 (+170205us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:47.127806 (+   279us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:20:47.127860 (+    54us) write_op.cc:312] APPLY: finished
0709 13:20:47.128184 (+   324us) log.cc:844] Serialized 81 byte log entry
0709 13:20:47.141308 (+ 13124us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:47.141434 (+   126us) write_op.cc:454] Released schema lock
0709 13:20:47.141482 (+    48us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:47.141913 (+   431us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:47.141941 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28625,"reported_tablets":37,"spinlock_wait_cycles":2813824,"child_traces":[["op",{"apply.queue_time_us":212997,"apply.run_cpu_time_us":14598,"apply.run_wall_time_us":223451,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":197664,"prepare.run_cpu_time_us":4120,"prepare.run_wall_time_us":12932,"replication_time_us":293282,"thread_start_us":284384,"threads_started":2}]]}
W20260709 13:20:52.115038 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 60805.1us
I20260709 13:20:52.021689 23809 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:52.120383 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:52.125814 16664 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:52.132542 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:52.020495 23808 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:52.135684 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.016721 23646 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.092265 23656 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:52.153615 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.077350 23707 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:20:52.159592 16592 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 12. Current term is 13. Ops: []
I20260709 13:20:52.138010 23814 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:20:52.160840 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 81790.1us
I20260709 13:20:52.141996 23811 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:52.147042 23656 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:52.164944 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.149972 23595 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:20:52.237323 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:52.159222 23757 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:20:52.247052 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 82161.2us
I20260709 13:20:52.247947 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.164611 23740 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:20:52.166705 23804 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:52.163314 23809 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:52.255733 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.272523 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.277370 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.273138 23804 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:52.283402 23694 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:52.286602 23335 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:52.290145 23757 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:52.246783 23595 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.291824 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:52.292829 23694 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:52.293507 23595 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:52.294338 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:52.295655 23814 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:52.296922 17414 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:52.297323 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:52.302541 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:52.303009 23814 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:52.305919 23814 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:52.307436 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:52.309103 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.311147 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.312230 23628 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:52.313937 23834 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:20:52.315975 23833 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:52.316881 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.317055 23299 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:52.318259 23740 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:52.318715 23740 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:52.319586 23740 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.320894 23646 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:52.323940 23692 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.325134 17409 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:52.325608 16593 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:52.326205 23692 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:52.327143 23809 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.327625 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.329727 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.252733 23693 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:52.315991 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.334990 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.252092 23758 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:52.339421 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:52.338127 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.336679 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.341295 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.343444 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.343987 16588 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:52.345606 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.346581 16593 raft_consensus.cc:2410] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:20:52.349704 16590 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:52.352053 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.352905 16589 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:52.354885 16593 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:20:52.356464 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.358806 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.360086 16588 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:20:52.360805 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.361848 16591 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:52.362072 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:52.364589 17414 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:20:52.366642 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.366911 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.367481 16588 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:52.368310 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.369027 16587 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:52.424072 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.434284 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:52.424499 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 45880.8us
I20260709 13:20:52.427770 16589 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:52.437713 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.439445 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.440958 16593 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
W20260709 13:20:52.452920 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 395119us
I20260709 13:20:52.342269 23815 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:20:52.415715 23577 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:52.466513 16742 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:52.466962 16589 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:20:52.460299 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:52.463268 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.466612 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.470398 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.470443 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.457278 16664 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:20:52.481606 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:52.497993 16596 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:52.500089 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.485702 16588 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:52.502171 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.505968 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.511233 23040 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:52.511659 23040 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:52.512049 23040 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:52.514056 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49553.5us
I20260709 13:20:52.517457 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.501408 23717 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:52.518389 16588 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 11.
I20260709 13:20:52.462433 23784 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:52.517529 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:52.516125 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.539649 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:52.540751 16742 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:52.541553 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.501073 23730 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:52.507592 23726 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:52.503535 23604 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:52.523276 23694 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:20:52.537858 23814 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:20:52.543601 23808 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:52.551457 23828 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:20:52.508567 23707 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:52.572152 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.572270 16634 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:52.502010 23577 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:52.570075 23814 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:20:52.518685 23806 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:52.583686 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.556888 23717 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:52.517256 23742 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:20:52.539654 23335 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
W20260709 13:20:52.638360 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57078.9us
I20260709 13:20:52.577206 23783 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:20:52.574424 23692 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:52.641131 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 59966.9us
W20260709 13:20:52.640724 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 68548.1us
I20260709 13:20:52.739851 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:52.586433 23730 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:52.724838 23707 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:52.753769 23634 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:20:52.749485 23741 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
W20260709 13:20:52.768124 23835 log.cc:927] Time spent T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:20:52.659209 23775 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:52.661253 23577 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:52.725090 16743 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 8 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:52.777580 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 271678us
I20260709 13:20:52.784224 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:52.786588 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.789624 17412 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:20:52.791249 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.791538 17411 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:20:52.792656 17414 raft_consensus.cc:2368] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20.
I20260709 13:20:52.668283 23656 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.794327 17409 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:52.797672 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.659541 23784 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 13:20:52.599584 23795 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.199s
I20260709 13:20:52.801501 23806 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:20:52.778949 23758 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:52.802770 17415 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:52.801880 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:52.776521 23717 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:52.815959 17413 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:52.788403 23707 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:52.794215 23605 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:52.809957 23775 raft_consensus.cc:697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:52.800722 23577 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:52.824357 23656 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:52.825989 23775 consensus_queue.cc:237] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:52.815951 23717 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:52.824204 23707 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.830029 23717 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:52.669965 23040 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.845690 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:52.810917 23758 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:52.816236 23737 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:52.803263 23694 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:20:52.856467 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:52.857362 16590 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:52.852350 23605 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:52.847869 23809 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:20:52.871773 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 125648us
I20260709 13:20:52.888489 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:52.858655 23577 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:20:52.860482 23807 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:20:52.893029 23788 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:52.894680 23788 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:52.895092 23788 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:52.895684 23040 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:52.882145 23605 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:52.906040 16561 leader_election.cc:400] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20.
I20260709 13:20:52.907912 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:52.909165 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:52.911480 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:52.882980 23784 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:20:52.885551 23758 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:52.949023 16736 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:20:52.927263 23510 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:52.968693 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:52.973618 17411 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:52.917012 23814 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:20:52.971349 22537 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:20:52.981211 22537 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:52.942232 23656 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:20:52.929148 23489 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:20:52.967909 23510 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:52.983127 22537 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:52.941905 23808 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:20:53.016389 22537 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:20:53.017032 17411 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:53.021032 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 93113.2us
I20260709 13:20:52.974579 23707 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:20:53.029080 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:53.056502 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45334.4us
I20260709 13:20:53.025674 23615 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:52.978816 23694 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:20:53.088474 23809 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:52.980752 23605 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.077998 23740 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.066299 23510 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:53.098594 23849 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:20:53.098712 23744 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:53.099552 23811 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:53.100499 23774 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:53.111126 23744 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:53.113631 23510 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.113732 23774 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:53.114774 23774 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.115751 23335 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:53.112291 23811 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:53.116319 23335 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:53.116912 23811 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.117204 23335 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.118341 23811 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:53.115789 23774 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:20:53.119319 23843 log.cc:927] Time spent T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:20:53.114202 23744 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.118407 23335 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:53.079183 23615 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.091176 23694 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:20:53.089188 23809 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.121024 23715 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:53.131335 23715 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.126296 23717 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.134298 23715 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.135577 23717 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.130249 23615 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.130441 23788 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.128449 23694 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:20:53.136778 23744 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:53.204679 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 110165us
W20260709 13:20:53.215291 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 185272us
I20260709 13:20:53.216231 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:53.226341 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1537) took 1167 ms. Trace:
I20260709 13:20:53.226792 23715 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:53.230113 23741 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:20:53.242172 23742 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:53.253861 23723 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.254462 17420 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 13:20:53.249366 16671 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:20:53.273257 23715 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.291344 23694 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.292622 23490 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.249176 17413 rpcz_store.cc:276] 0709 13:20:52.059171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:52.060407 (+  1236us) service_pool.cc:224] Handling call
0709 13:20:52.060658 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:52.816548 (+755890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:53.225630 (+409082us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b540016c454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:53.226046 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736503040}
I20260709 13:20:53.294312 23489 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.297785 23783 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:53.298352 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.299917 16665 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:53.302520 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.258003 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:53.256421 23742 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:53.323227 23737 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:53.305671 23848 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.224s	user 0.003s	sys 0.000s
I20260709 13:20:53.326481 23595 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:53.318307 23503 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.310457 23490 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.331696 23489 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.328310 17420 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:53.309854 23694 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.308079 23788 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.306429 23723 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.346838 23871 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.385264 23723 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.323853 23857 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.335562 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:53.384796 23871 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.386554 23857 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.389562 23857 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.353250 16662 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:53.359750 16667 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:53.362404 23742 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:53.387603 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52516.2us
I20260709 13:20:53.393455 23742 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:53.274693 23758 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:53.372069 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.363139 23737 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.373899 23595 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.397706 23737 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.399412 23847 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:53.374377 23503 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.400841 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:53.401651 23335 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.402585 23335 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.374526 23735 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.404174 23735 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.404974 23335 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.374210 17415 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:20:53.378906 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.378779 23489 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:53.408676 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.423586 16662 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:53.381059 23809 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.420506 23788 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:53.425016 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:53.424824 23788 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.371096 23741 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:53.352293 23783 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:53.426817 23741 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.427822 23783 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.430042 23783 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:53.376641 23490 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.431448 23503 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.406489 23735 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.381763 23694 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.414408 23737 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:53.375367 23636 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:53.439464 16665 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:20:53.437280 23737 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.445017 23621 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:53.447356 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.448195 16595 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:20:53.449486 23869 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.450312 23869 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.451274 23788 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.448243 23595 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.447156 23621 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.456840 23636 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.458984 23621 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.459357 23636 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.455767 23869 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.456055 23503 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.491060 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:53.490366 16667 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:20:53.495034 16664 raft_consensus.cc:2393] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:53.496460 23869 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:53.497407 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:53.497648 23869 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:53.498123 23869 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:53.496763 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.499612 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.505825 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:53.508092 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 292165us
I20260709 13:20:53.500934 23871 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.519606 23299 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:53.519858 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.520404 23299 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.499435 23503 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.525287 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:53.528038 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:20:53.532053 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.527576 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.548560 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.549329 16663 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:53.550050 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.552307 23621 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:53.553146 23621 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:53.555017 23870 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:20:53.558938 23869 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.560709 23869 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:53.560868 23595 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.561590 23595 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.561962 23517 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.563328 23517 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.563556 23694 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:53.567544 23694 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:53.567925 23694 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:53.569875 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.570214 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.571116 16745 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 17: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:53.571568 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.572956 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:53.575830 16742 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:53.576973 16738 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:20:53.577343 23737 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.569146 23694 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.577895 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.578138 23737 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:53.579046 23694 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:53.578949 23737 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.580492 16595 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:53.581400 17413 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:53.582149 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.582839 16745 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:53.582835 16596 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:20:53.583117 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:53.583858 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.584169 16596 pending_rounds.cc:85] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 12
I20260709 13:20:53.584551 16596 pending_rounds.cc:107] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:20:53.543354 23715 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.584842 16596 raft_consensus.cc:2889] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:53.566582 23874 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:20:53.583259 23299 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.564810 23595 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.564559 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:53.526616 23871 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.593736 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.600201 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.602878 16592 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:53.603122 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.588711 23503 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.600541 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.606994 16745 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:53.608846 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:53.611796 16587 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:53.611994 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.609026 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.613134 23873 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:53.613762 23873 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.615060 23517 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.616147 23873 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.623970 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.626552 16595 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:53.626904 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.633663 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:53.638470 23621 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:53.640925 23811 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.003s
I20260709 13:20:53.641990 16594 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:20:53.642414 16594 pending_rounds.cc:85] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 10
I20260709 13:20:53.642642 16594 pending_rounds.cc:107] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:20:53.642930 16594 raft_consensus.cc:2889] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:20:53.647213 23744 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.503s	user 0.001s	sys 0.020s
I20260709 13:20:53.648136 23717 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.648732 23717 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.656098 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.658324 23299 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:53.676156 23717 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.684360 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.685878 23853 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:20:53.687294 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 122956us
W20260709 13:20:53.714468 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.688340 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:53.718070 23299 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:53.693302 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.695531 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.696727 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.697069 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.720216 16489 catalog_manager.cc:5697] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:53.701768 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:53.609902 23871 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.701656 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:53.722891 16489 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 16 to 17, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:53.727308 16489 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:53.703933 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:53.705297 23737 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:53.706270 16664 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Committing config change with OpId 14.11: config changed from index 8 to 11, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:20:53.704589 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 84134.1us
I20260709 13:20:53.719973 23853 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:53.699731 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.741037 16662 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:53.748844 23853 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:53.664556 23517 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:20:53.750913 23517 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:53.751978 23517 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.728168 23871 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.756754 23737 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.757609 23737 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.758430 23636 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.759025 23636 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.759469 23299 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:53.760857 17413 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:20:53.761106 23636 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.764204 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:53.765002 23744 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:53.765247 23737 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.765432 23744 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.766161 23744 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:53.775755 23717 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.776539 23717 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.780055 23717 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:53.780655 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62134.4us
I20260709 13:20:53.787614 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.784998 23744 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:53.791550 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.791767 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.794728 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.794983 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.795730 16671 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:20:53.796959 23744 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.797422 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:53.796989 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.800282 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:53.801903 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.802703 16591 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
W20260709 13:20:53.803550 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.804996 16742 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:53.806391 23853 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:53.806906 23853 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:53.807662 23853 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:53.808451 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.832244 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:53.846165 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.835570 16591 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
W20260709 13:20:53.856614 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:53.860869 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:53.809557 23741 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:53.862349 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:53.862986 23741 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.854581 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:53.871407 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.873121 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:53.874456 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:53.894793 16594 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:53.879930 16742 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:53.878631 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:53.902427 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:53.874787 23811 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:53.875675 23595 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:53.901857 23741 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:53.906952 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:53.877770 23809 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:53.891279 23490 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:20:53.919294 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58309.6us
I20260709 13:20:53.918207 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.915565 16489 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:53.905234 23811 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.920044 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:53.912515 23299 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:53.940840 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.944984 16745 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:53.944830 17414 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:20:53.953243 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:53.954223 16595 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:20:53.941531 16588 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:53.950179 16592 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:53.935825 23811 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:53.942657 23783 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:20:53.957688 23717 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:53.974632 23717 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.945895 23595 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:53.977646 23717 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.972401 23783 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:53.978936 23783 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:53.966471 16593 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:20:53.984752 17415 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:20:53.985961 16594 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:53.984292 23595 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:53.995716 16489 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:20:54.007948 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:54.022572 16561 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:54.042346 23489 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:20:54.053074 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.050843 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.058333 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.056202 16489 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 17 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:54.046257 23869 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:54.070508 16489 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 20 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:20:54.020725 23595 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:20:54.084677 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.090215 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.071890 23869 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.090649 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:54.088145 16738 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:20:54.103320 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:54.105469 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54796.9us
I20260709 13:20:54.100037 23869 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:54.109241 16489 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 17 to 18, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 18 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:20:54.123649 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 170287us
I20260709 13:20:54.119838 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.110257 23737 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.124399 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:54.127447 23656 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:54.127578 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.132635 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.133263 16591 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:54.134280 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.138470 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.139273 16593 raft_consensus.cc:2393] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:54.141386 16595 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:54.142412 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.144162 16634 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:54.144932 23886 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:20:54.154815 23737 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.156411 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.157362 23737 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:54.127578 23846 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:54.158274 23873 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:54.159169 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.161482 23621 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:54.151589 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.147015 23735 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:20:54.163411 23735 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.163815 23735 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:20:54.164702 23879 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:20:54.151805 23853 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.169414 16589 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:20:54.170361 23853 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:54.171535 23873 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:54.172119 17414 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:54.172524 23873 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.183528 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 233) took 7308 ms. Trace:
I20260709 13:20:54.189384 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.191509 23503 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:54.193742 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.194689 23723 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:54.195187 23723 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.195591 23723 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:54.161401 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.202682 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.203750 16662 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:54.205183 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.207168 23503 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:54.250350 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.240072 23335 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.255627 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.254307 23694 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:20:54.242910 23636 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:54.174736 23811 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.217293 23758 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:20:54.215893 23783 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.267009 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:54.266526 16587 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:54.184118 16492 rpcz_store.cc:276] 0709 13:20:46.884145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:46.888624 (+  4479us) service_pool.cc:224] Handling call
0709 13:20:54.183303 (+7294679us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:52.358302 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:52.511396 (+153094us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:52.511432 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:20:52.511450 (+    18us) tablet.cc:662] Decoding operations
0709 13:20:52.512297 (+   847us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:52.512358 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:52.512382 (+    24us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:20:52.512995 (+   613us) tablet.cc:701] Row locks acquired
0709 13:20:52.513013 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:20:52.513178 (+   165us) write_op.cc:270] Start()
0709 13:20:52.513300 (+   122us) write_op.cc:276] Timestamp: P: 1783603252513128 usec, L: 0
0709 13:20:52.513318 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:20:52.585299 (+ 71981us) log.cc:844] Serialized 2793 byte log entry
0709 13:20:53.143215 (+557916us) op_driver.cc:464] REPLICATION: finished
0709 13:20:53.153533 (+ 10318us) write_op.cc:301] APPLY: starting
0709 13:20:53.153638 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:53.389841 (+236203us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:53.389982 (+   141us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:53.441413 (+ 51431us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:53.506477 (+ 65064us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 13:20:53.512983 (+  6506us) write_op.cc:312] APPLY: finished
0709 13:20:53.667138 (+154155us) log.cc:844] Serialized 73 byte log entry
0709 13:20:53.668963 (+  1825us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:53.689006 (+ 20043us) write_op.cc:454] Released schema lock
0709 13:20:53.689077 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:53.692164 (+  3087us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:53.692195 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1755074,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":10022,"apply.run_cpu_time_us":15683,"apply.run_wall_time_us":538589,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33415,"num_ops":7,"prepare.queue_time_us":332872,"prepare.run_cpu_time_us":24501,"prepare.run_wall_time_us":502486,"raft.queue_time_us":127443,"raft.run_cpu_time_us":9021,"raft.run_wall_time_us":9989,"replication_time_us":629734,"spinlock_wait_cycles":78720,"thread_start_us":547499,"threads_started":4,"wal-append.queue_time_us":264806}]]}
I20260709 13:20:54.280965 23605 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.281733 23605 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.284206 23605 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:54.287122 23890 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:20:54.243194 23744 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
W20260709 13:20:54.290714 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85599.7us
I20260709 13:20:54.291741 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.292605 23783 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:54.293082 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:54.293810 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.293974 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.295053 23809 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:54.295845 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.292912 23490 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:20:54.298990 23595 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:54.299780 23595 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:20:54.300151 23595 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:54.300941 23717 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:54.301412 23717 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.301889 23717 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:54.297591 23788 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:54.240927 23621 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:20:54.305896 23621 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:54.306430 23621 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:54.294420 23853 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:20:54.298183 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:54.308808 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.308889 23595 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:54.266721 23656 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.314391 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:54.298271 23809 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:54.309475 23853 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:20:54.323007 17376 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:54.326185 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83404.5us
I20260709 13:20:54.319940 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.337774 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:54.339224 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:54.340966 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.344200 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.346349 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.348860 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.352597 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:54.313825 23788 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:54.357630 23299 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:20:54.352636 17415 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:20:54.362663 23490 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:20:54.363744 23869 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.365118 16589 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:54.325753 23715 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:20:54.366761 23715 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:54.368590 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:54.370471 23811 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:54.372920 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:54.373708 16743 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:20:54.328084 23890 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:20:54.379029 23869 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:54.325425 23853 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:54.381600 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:54.381335 23853 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.385291 23735 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.386591 23758 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.353127 23656 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.387355 23758 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.388898 23656 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:54.389700 23758 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:54.390782 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51675.7us
I20260709 13:20:54.390885 23737 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:20:54.391752 23490 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:54.391812 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:54.392692 16738 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:54.392935 23490 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.396143 23621 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.397641 23621 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:54.402158 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:54.407200 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:54.408032 16664 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:54.409226 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.408478 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:54.416795 16743 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:20:54.419386 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:54.382731 23853 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:54.444113 16738 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:20:54.446925 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:20:54.453293 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44893.4us
I20260709 13:20:54.408147 23890 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:54.454365 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:54.458966 16671 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:20:54.465278 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 220123us
W20260709 13:20:54.472415 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52970us
I20260709 13:20:54.473268 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:54.472970 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:54.477473 16593 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:54.451427 23517 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:54.457969 23869 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:20:54.479701 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.456377 23503 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.483475 23517 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.481257 23715 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:54.456248 23788 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:20:54.458667 23694 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:20:54.542349 16593 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
W20260709 13:20:54.536074 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 77550.1us
I20260709 13:20:54.500404 23897 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.472596 23846 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 13:20:54.546746 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 68243.3us
I20260709 13:20:54.548723 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.483798 23723 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.542874 23517 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:54.544199 23490 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:54.548741 23758 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:20:54.573546 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 100356us
I20260709 13:20:54.548820 23869 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:54.564482 23857 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:54.582346 17420 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:20:54.602630 23735 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:54.608858 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 285995us
I20260709 13:20:54.611541 23723 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:54.554809 23503 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.560155 23897 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:54.706238 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 226603us
W20260709 13:20:54.719240 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 169986us
I20260709 13:20:54.722002 23897 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:54.722620 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:54.726047 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:54.726332 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:54.726759 16663 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:54.726995 16664 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:20:54.738270 23656 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.738732 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:54.739004 23656 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.739190 23595 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.739578 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:54.741309 23656 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:54.741997 23715 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:54.742547 23715 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:54.743943 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 126314us
I20260709 13:20:54.745405 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:54.742038 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:54.749715 16667 raft_consensus.cc:2368] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23.
I20260709 13:20:54.607439 23869 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.748467 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:54.774657 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:54.775349 16596 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:54.778909 23857 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:54.779454 23899 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:20:54.779485 23857 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:54.782387 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:54.783049 16591 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:20:54.748556 23844 log.cc:927] Time spent T 00000000000000000000000000000000 P bfe27eaae9f4449e96031a0b1485f80c: Append to log took a long time: real 0.280s	user 0.001s	sys 0.001s
I20260709 13:20:54.741482 23595 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:54.790843 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 45536.6us
W20260709 13:20:54.791463 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 232921us
I20260709 13:20:54.794504 17414 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:54.794662 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.799504 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:54.802572 23605 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:54.803723 23918 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:54.732923 23489 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:54.809707 16596 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:54.810195 23811 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:54.794488 23758 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:54.785630 23299 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:54.850700 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.830473 23605 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:20:54.854696 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 135268us
I20260709 13:20:54.869673 16708 leader_election.cc:400] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23.
I20260709 13:20:54.873731 16708 leader_election.cc:403] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:20:54.871431 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.885413 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:54.862323 23605 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:54.871523 23783 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:54.847528 23489 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.847587 23811 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.887599 16634 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:20:54.899730 23901 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:20:54.899654 23914 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.897462 23910 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:20:54.853102 23918 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.861459 23299 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.899411 23912 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:20:54.849145 23758 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:54.910645 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 43619.4us
W20260709 13:20:54.908435 23717 consensus_meta.cc:287] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.001s
I20260709 13:20:54.893744 23809 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23.
W20260709 13:20:54.930011 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 82096.8us
W20260709 13:20:54.929526 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47668.4us
I20260709 13:20:54.931216 17374 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:54.937191 23595 raft_consensus.cc:2804] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:54.937883 23595 raft_consensus.cc:697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:54.939122 23595 consensus_queue.cc:237] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.941901 23915 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:54.942616 23915 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:54.946079 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:54.946671 16665 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:54.908183 23890 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:54.948496 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:54.930403 23912 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:54.935179 23783 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:20:54.991896 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:54.992317 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 106060us
I20260709 13:20:54.994814 16737 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:20:55.004367 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68757.7us
I20260709 13:20:54.977111 23694 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:54.978188 23809 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:54.969204 23915 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:54.947962 23717 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:54.977051 23912 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.037434 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.043797 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:54.961833 23517 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.006304 23783 raft_consensus.cc:697] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:55.021785 23503 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:54.948282 23914 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.026975 23694 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.083887 16591 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:20:55.027699 23809 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:55.082784 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:54.978942 23923 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:20:55.082049 17409 raft_consensus.cc:1240] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:20:55.086805 16589 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 13:20:55.100255 23902 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:20:55.103438 23783 consensus_queue.cc:237] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.104501 23918 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.086251 23917 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:55.090063 23694 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:55.105798 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 100587us
W20260709 13:20:55.153039 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109471us
I20260709 13:20:55.161387 23758 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:55.162483 23900 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.455s	user 0.001s	sys 0.002s
W20260709 13:20:55.163085 23926 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:20:55.163314 23788 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.114892 23897 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:55.164007 23788 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.105329 23809 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.165701 23809 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:55.166407 23788 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.165025 23717 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.158882 23917 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.171829 23299 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.147970 23735 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:55.170970 23715 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.176373 23517 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.180313 23914 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:55.181732 23935 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:20:55.218161 23924 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.168s	user 0.002s	sys 0.000s
I20260709 13:20:55.187229 23811 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.162849 23489 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.191821 23890 consensus_queue.cc:1059] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:20:55.196722 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 330941us
W20260709 13:20:55.152793 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 183411us
W20260709 13:20:55.205132 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 122500us
W20260709 13:20:55.212821 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 54191.6us
W20260709 13:20:55.223839 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 185505us
I20260709 13:20:55.226511 17374 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:55.235169 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.236131 16593 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:55.237211 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:55.230967 23723 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:55.258445 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.260802 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.261554 17411 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:55.262513 23927 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.263118 23927 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.265195 23912 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.263458 16596 raft_consensus.cc:1240] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:20:55.270980 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.271299 23890 raft_consensus.cc:3055] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:55.271497 23912 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.271785 23890 raft_consensus.cc:740] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:55.273168 16592 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 4973) took 1071 ms. Trace:
I20260709 13:20:55.272953 23890 consensus_queue.cc:260] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.274228 23890 raft_consensus.cc:3060] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.273536 16592 rpcz_store.cc:276] 0709 13:20:54.201286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.201417 (+   131us) service_pool.cc:224] Handling call
0709 13:20:54.201735 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:54.203186 (+  1451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:54.203234 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:54.203796 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:20:54.457907 (+254111us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:20:54.457954 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:54.458029 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:55.161521 (+703492us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:55.272853 (+111332us) raft_consensus.cc:1685] finished
0709 13:20:55.272875 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:55.273090 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":49516288,"thread_start_us":252231,"threads_started":1,"wal-append.queue_time_us":343547}
I20260709 13:20:55.213249 23737 consensus_queue.cc:1059] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:20:55.275373 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.269855 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.276520 17414 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:55.277709 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49651.5us
I20260709 13:20:55.279448 23940 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:55.280254 23940 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.282168 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.280478 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.282886 23925 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:55.283547 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.283463 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.284690 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.279392 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.287011 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.287870 16737 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:55.289814 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.290534 16743 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:55.292711 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.293438 16744 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:55.297667 23912 raft_consensus.cc:3055] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:55.298151 23912 raft_consensus.cc:740] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:55.299218 23912 consensus_queue.cc:260] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.300531 23912 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.303668 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:55.304272 17411 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.308238 23715 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.310425 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:55.238384 23517 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.311902 23927 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.311597 23517 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.270085 23723 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:55.314199 23517 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.313494 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.314131 23723 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.315405 23927 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.315915 23927 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.272060 17416 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:55.299786 23906 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:55.271862 17412 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:20:55.306658 23857 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.527s	user 0.005s	sys 0.003s
I20260709 13:20:55.319751 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.325559 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.323520 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:55.327724 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59652.6us
W20260709 13:20:55.329838 23928 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:20:55.334354 23914 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:55.336090 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.318655 23694 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.336807 23914 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.337277 23914 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.339005 23694 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.339533 23920 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:55.339643 23656 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.340137 23920 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.342029 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.342922 16742 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
W20260709 13:20:55.337304 23904 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.726s	user 0.000s	sys 0.003s
I20260709 13:20:55.282580 23940 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.351492 23907 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.324476 23517 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:55.342325 23920 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.352172 23907 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:55.352847 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 122349us
I20260709 13:20:55.354035 17409 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:55.354101 23605 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.356534 16741 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18.
I20260709 13:20:55.356961 23605 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.358636 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:55.326244 23906 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.359401 17413 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:20:55.363051 23905 log.cc:927] Time spent T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.761s	user 0.003s	sys 0.000s
I20260709 13:20:55.332966 23857 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.368409 17376 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.369791 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.370767 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.372344 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.373889 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.376085 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.376847 16743 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:55.378302 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:55.382849 17411 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:55.383435 16739 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:20:55.385283 17413 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:55.386125 16596 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:55.386759 17376 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18.
I20260709 13:20:55.404255 17376 leader_election.cc:403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:55.388329 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.388523 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.388744 16737 raft_consensus.cc:1275] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:20:55.388957 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.388329 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.384951 23595 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:55.405246 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.390002 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:55.407349 16667 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:20:55.390475 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.391068 23907 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.392791 23857 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.393579 23906 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.395030 16740 raft_consensus.cc:1240] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 15. Current term is 16. Ops: []
I20260709 13:20:55.384603 23915 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:55.387053 23503 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.397224 23927 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.398785 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:55.405154 16736 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:55.436623 16592 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
W20260709 13:20:55.393603 23932 log.cc:927] Time spent T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:20:55.439008 23927 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:55.439193 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.440006 16737 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:55.440068 23913 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.441046 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.440822 23913 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.441749 16741 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:55.447113 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.448288 23744 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:55.448819 23744 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.451349 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.352274 23914 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.459820 23914 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.461925 23919 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:20:55.463205 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.463847 16663 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:55.416221 23715 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18.
I20260709 13:20:55.416158 23595 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:55.464710 23595 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.400418 23335 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:55.466621 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.467239 16664 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:55.283504 23925 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.476145 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.470311 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.421465 23737 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.482522 16663 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:55.450752 23744 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.488377 16738 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:55.423806 23915 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.502784 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.483029 23335 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:55.517923 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:55.499230 23915 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.520738 16667 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
W20260709 13:20:55.521772 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 123099us
I20260709 13:20:55.522478 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.509284 23335 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.524421 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.524677 23335 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.526618 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:55.527119 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.528268 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:55.528769 16671 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.529429 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.530745 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.531231 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.531885 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:55.532388 16663 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.532648 16670 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:55.498984 23490 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.524452 16665 raft_consensus.cc:1240] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 14. Current term is 15. Ops: []
I20260709 13:20:55.535113 16664 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:55.548341 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:55.551687 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.553936 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.557152 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:55.497519 23737 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.569325 16736 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:55.575184 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.578966 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.557224 23869 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.560438 23489 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.550283 23913 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.583590 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.578047 23912 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:20:55.590324 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.591326 16741 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:20:55.593288 16736 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:55.588830 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:55.564365 23809 consensus_queue.cc:1059] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:20:55.558112 23490 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.613210 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.614135 23951 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:20:55.614419 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:55.614935 23948 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.615461 23948 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.617532 23948 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.619138 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.622027 23947 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:55.622037 16663 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:55.622900 23947 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:55.624260 23947 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.628319 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.640426 16736 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:55.591009 23715 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:20:55.650126 23955 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:20:55.651594 23873 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:55.654196 16671 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:55.652237 23873 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:55.608093 23946 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.175s
I20260709 13:20:55.660295 23873 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.587951 23910 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.675891 16636 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:55.681636 16636 leader_election.cc:403] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:55.680478 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.672209 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:55.683408 16587 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:55.687630 16710 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.688663 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.660323 23960 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:55.691890 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:55.701239 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.702641 23853 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:55.703781 23299 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:55.705163 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.705859 16739 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.703807 23925 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.707255 23503 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:55.708222 23503 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:55.708415 23873 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.708724 23503 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.708935 16736 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:20:55.709293 23503 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:55.717166 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.671571 23869 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:55.720979 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:55.680579 23951 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:55.671319 23913 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.719342 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:55.732913 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:55.684589 23933 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:55.734087 16668 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:20:55.740453 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:20:55.743001 16744 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:55.680186 23621 consensus_queue.cc:1048] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:20:55.744562 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:55.680153 23595 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:55.749871 23961 log.cc:927] Time spent T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:20:55.750073 23735 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:55.750525 23735 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:55.750959 23735 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:55.751551 23929 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:55.752329 23929 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:55.753028 23871 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:20:55.737638 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 148889us
W20260709 13:20:55.738956 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66893.6us
I20260709 13:20:55.754014 23490 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.755867 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.734211 23869 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.758344 23595 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:55.683548 23910 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.764259 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:55.762985 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 71150.7us
I20260709 13:20:55.764259 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:55.768167 16736 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:55.767105 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:20:55.761220 23962 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:20:55.747330 23788 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.758720 23869 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:55.785209 16739 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:55.784617 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:55.743719 23913 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.768965 23929 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:20:55.907022 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 122476us
I20260709 13:20:55.825989 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:55.774667 23335 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.920724 23869 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:20:55.921222 23869 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:20:55.816664 23938 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.625s	user 0.000s	sys 0.005s
I20260709 13:20:55.921661 23869 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:20:55.885521 23715 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:55.830367 23489 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:20:55.898792 23890 consensus_meta.cc:287] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.002s
I20260709 13:20:55.784355 23927 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.932643 23927 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.933101 23913 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.879004 23694 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:55.784065 23717 consensus_queue.cc:1059] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.150s
I20260709 13:20:55.884375 23960 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:55.875116 23898 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:55.787964 23853 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:55.941936 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.934121 23694 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:55.818104 23906 raft_consensus.cc:3055] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:20:55.787395 23948 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:55.934544 23717 raft_consensus.cc:3055] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:20:55.922591 23335 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:55.944983 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 204654us
W20260709 13:20:55.960582 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 180903us
W20260709 13:20:55.962751 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:55.942844 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.987793 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.960005 23906 raft_consensus.cc:740] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:55.976722 23927 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.931090 23715 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:55.978986 23737 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.800591 23788 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.805665 23942 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.217s
I20260709 13:20:55.877516 23503 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.001566 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.832077 23910 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:55.990691 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:55.805871 23911 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.241s
I20260709 13:20:56.018567 16671 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:55.943955 23898 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:55.944520 23960 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:55.997052 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.024503 16739 raft_consensus.cc:2368] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:55.997660 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.998219 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:55.999253 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:56.027626 16664 raft_consensus.cc:2368] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:55.999955 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.003041 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:56.029167 16596 raft_consensus.cc:2368] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:56.030267 17420 raft_consensus.cc:2368] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:20:55.981714 23717 raft_consensus.cc:740] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:55.998309 23913 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:56.011673 23735 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.031454 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.032339 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:55.998066 23737 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:56.033733 16710 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:56.034672 16739 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.034525 16587 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:56.035012 17409 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:55.821668 23951 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:56.025779 23888 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.013s	sys 0.043s
I20260709 13:20:56.024677 23907 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.041889 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:56.070370 17416 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.070240 23907 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.072443 23907 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:56.073499 16739 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:56.048403 23503 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:56.047879 23737 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.053207 23906 consensus_queue.cc:260] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:56.053951 23915 consensus_meta.cc:287] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.003s
W20260709 13:20:56.044240 23888 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.443s	user 0.014s	sys 0.043s
I20260709 13:20:56.056102 23853 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.080261 23915 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.081445 23737 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.082278 23621 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.082456 23915 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.058600 23735 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:56.068001 23694 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:56.088385 17416 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:56.094638 16559 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:56.095657 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.096415 17376 leader_election.cc:400] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:56.096742 17376 leader_election.cc:403] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.069633 23960 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:56.130262 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:56.060189 23717 consensus_queue.cc:260] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.038007 23913 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.166939 16744 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:56.076620 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.076756 23906 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:20:56.084816 16595 raft_consensus.cc:2368] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19.
I20260709 13:20:56.111430 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.111912 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.207489 16668 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:56.207677 16671 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:56.127750 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.132309 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.133258 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:56.129010 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.167428 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.210695 16739 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.134693 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:56.167872 17412 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
W20260709 13:20:56.168347 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 72069.9us
I20260709 13:20:56.043958 23910 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:20:56.203759 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:56.215675 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 81020.6us
I20260709 13:20:56.222359 16589 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:56.226775 16739 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:56.229769 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:56.233681 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.234304 16591 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:56.203521 23694 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:56.235215 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.075145 23869 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.232287 17376 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:56.083426 23621 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.219729 23960 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:56.219414 23948 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:56.241125 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.242520 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.223713 23489 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:56.245131 16669 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:56.247391 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:56.166973 23517 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:20:56.231740 23717 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:20:56.242540 23656 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.252420 17413 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:56.253536 16588 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:20:56.257896 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.245042 23951 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:56.263993 23490 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:20:56.253458 23948 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.246181 23694 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:56.267234 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.251961 23595 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:56.263993 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:56.270174 17374 leader_election.cc:400] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:56.274019 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.271294 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.275338 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:56.274240 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.168697 23898 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.257143 23489 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:56.262446 23912 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:20:56.297734 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.299208 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.306156 16744 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:56.296875 16710 leader_election.cc:400] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:20:56.312309 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.283411 16739 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:56.316690 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.283955 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.319258 16710 leader_election.cc:403] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.319031 17410 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:56.323921 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.324553 16666 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:56.294269 23788 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.300101 23621 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.325325 23605 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18.
I20260709 13:20:56.298697 23737 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.265758 23656 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.330103 23656 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:56.323944 23489 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.332705 23929 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.333177 23929 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.333488 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:56.285795 23976 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:20:56.322444 23595 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:20:56.336200 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 72285.1us
I20260709 13:20:56.336392 23595 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.337366 23717 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:56.337958 23717 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:56.291419 23940 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:56.339382 23940 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:56.339788 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.293601 23913 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.340466 16670 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:56.342586 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:56.256462 23919 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:20:56.272251 23948 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.346799 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:56.356731 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:56.357470 16739 raft_consensus.cc:2410] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:20:56.357564 16663 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.357892 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:56.267190 23951 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.375615 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.377194 16663 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:20:56.375979 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:56.267367 23925 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 13:20:56.322244 23909 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:56.391808 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.384478 16745 raft_consensus.cc:1275] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:20:56.402530 16745 pending_rounds.cc:85] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 14
I20260709 13:20:56.395499 16596 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:20:56.399320 16590 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:20:56.355726 23517 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:56.402819 16745 pending_rounds.cc:107] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:20:56.400971 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:20:56.408705 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38582 (request call id 3476) took 1033 ms. Trace:
I20260709 13:20:56.420802 16745 raft_consensus.cc:2889] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:56.409246 23869 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.404456 23605 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:56.371245 23918 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:20:56.294550 23694 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.411653 16741 raft_consensus.cc:1275] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:20:56.293774 23715 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.428845 16741 pending_rounds.cc:85] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 9
I20260709 13:20:56.429217 16741 pending_rounds.cc:107] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:20:56.429034 23846 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:56.429453 16741 raft_consensus.cc:2889] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:20:56.430348 23715 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:56.321597 23969 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 13:20:56.417904 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:56.431056 17415 raft_consensus.cc:1240] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:20:56.431258 16669 raft_consensus.cc:606] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:56.431808 16669 raft_consensus.cc:614] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:56.420938 16595 rpcz_store.cc:276] 0709 13:20:55.375063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:55.375260 (+   197us) service_pool.cc:224] Handling call
0709 13:20:56.408467 (+1033207us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b540010d454. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:20:56.408629 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1558088576}
I20260709 13:20:56.360656 23933 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:20:56.426105 23517 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.403854 23947 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.439196 23947 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.440688 23517 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.441720 23947 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:56.338851 23717 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.378696 23967 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:56.384300 23853 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:56.390532 23951 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:56.452221 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51365.7us
I20260709 13:20:56.455098 23605 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:20:56.454777 23788 consensus_queue.cc:1059] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:56.463155 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:56.465412 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.477061 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.457293 23853 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.490748 23980 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.491439 23980 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.493712 23980 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:56.502202 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.503551 16667 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:56.504516 16667 raft_consensus.cc:606] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.505152 16667 raft_consensus.cc:614] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:56.493275 23788 raft_consensus.cc:3055] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:20:56.474514 23946 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:20:56.502270 23853 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:20:56.540920 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 214823us
I20260709 13:20:56.460395 23967 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.551924 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.567866 17416 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:56.569129 23967 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:56.555928 17374 leader_election.cc:400] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19.
I20260709 13:20:56.516405 23788 raft_consensus.cc:740] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:56.593966 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:56.594977 16666 raft_consensus.cc:606] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.595474 16666 raft_consensus.cc:614] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:20:56.598423 17414 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:56.597554 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.604633 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.588822 23809 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:56.619925 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.609019 23595 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:20:56.625051 16592 raft_consensus.cc:2410] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:20:56.624607 16636 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 15. Current term is 16.
I20260709 13:20:56.625150 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:56.623592 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:56.621506 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:56.608819 23335 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:20:56.640653 23929 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.643635 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.635458 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:56.661057 23853 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.651262 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:56.685246 16745 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:56.685508 16740 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:56.617830 23809 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.660189 23621 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:56.697165 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:20:56.698212 23694 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:56.698653 23694 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:56.692410 23873 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:56.708285 23941 consensus_queue.cc:1048] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:56.699054 23694 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:56.661336 23788 consensus_queue.cc:260] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:56.747411 23987 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:20:56.709832 23873 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.748380 23809 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.755620 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:56.756320 16591 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:20:56.704805 23948 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.746276 23951 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.797705 23951 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:56.771021 23853 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.752743 23873 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.803378 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:56.777376 23929 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:56.779251 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:56.805119 16665 raft_consensus.cc:606] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.783259 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:56.765533 23940 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.807325 23940 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:56.817353 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:20:56.818123 16665 raft_consensus.cc:606] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.821686 23948 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:56.839840 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:20:56.904157 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:56.899322 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.901041 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:56.916004 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:20:56.916447 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:56.902334 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:56.924628 16739 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:20:56.918506 16742 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
I20260709 13:20:56.919507 17411 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:20:56.834921 23694 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:56.921458 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:56.922773 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:20:56.902935 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:20:56.929209 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:56.937093 17414 raft_consensus.cc:2393] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:56.920444 16665 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.932096 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:56.942710 16596 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.884379 23898 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:56.958336 16665 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:56.878767 23948 consensus_queue.cc:1048] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:20:56.953068 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:56.963680 17411 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:20:56.963907 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:56.954288 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:56.964730 17415 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.935397 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:20:56.956952 17420 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 12: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:56.934805 16708 leader_election.cc:400] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19.
I20260709 13:20:56.970590 16708 leader_election.cc:403] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:20:56.960557 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:56.961292 16590 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:56.971899 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:56.973884 17414 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:56.974483 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:56.983165 23299 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:56.983803 23299 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:56.984207 23299 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:56.918196 23873 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:56.979811 23503 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:20:56.989975 17419 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:56.996551 23694 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:57.006372 23907 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:56.971961 23869 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:56.969435 23919 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:57.004526 23873 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:56.987546 23853 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) with higher term. Message: Invalid argument: T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19.
I20260709 13:20:57.077623 16736 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:20:57.083667 16596 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:20:57.092677 17415 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:57.040727 23694 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:57.026518 23931 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:20:57.026343 23910 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:20:57.095772 16710 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:57.104233 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 100467us
I20260709 13:20:57.109864 23951 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:20:57.127437 16596 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:57.101042 23694 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:57.140872 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 162754us
I20260709 13:20:57.132933 23788 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:57.144263 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:57.149863 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:57.133462 23951 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:57.175822 16662 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:57.176542 23951 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.177325 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:57.181427 23694 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:57.158957 23948 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:20:57.161514 24001 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:57.172465 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:57.183333 23783 raft_consensus.cc:993] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:57.183233 24001 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.183929 23783 raft_consensus.cc:1081] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:57.174228 16664 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:20:57.187584 23989 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.190s	user 0.003s	sys 0.000s
I20260709 13:20:57.190536 16662 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:20:57.194885 16671 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:20:57.196835 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:57.199934 23299 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.206107 16671 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:57.209771 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:57.212934 16671 raft_consensus.cc:2393] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:57.214224 23717 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:57.216336 16665 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:57.216655 23717 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:20:57.217641 23942 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:20:57.218551 23942 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:20:57.218938 23942 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:57.220181 23942 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:57.223311 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 73525.4us
I20260709 13:20:57.224653 23299 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:57.232050 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:57.240869 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:57.240908 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:57.242221 16591 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:20:57.243090 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:57.243871 16665 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:57.246593 17412 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:57.253790 23960 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.254578 23960 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.255486 24000 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.256026 24000 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.258284 24000 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:57.260977 23898 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:20:57.262239 24001 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:57.264012 16671 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:57.265120 23960 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:57.277745 16593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:20:57.277899 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:57.278509 16593 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:57.279114 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:57.280531 16593 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:57.258193 16591 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:57.302443 23299 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:57.310590 23299 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:57.304219 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:57.309144 16665 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:20:57.324534 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:57.336150 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:57.341452 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:57.341209 16593 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:57.342237 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:57.344846 16736 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:57.344946 23299 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:57.347288 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:57.315284 23869 raft_consensus.cc:2764] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:20:57.349776 16740 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:57.217676 23717 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.352452 23942 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:57.353994 23873 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:57.317348 23694 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:20:57.364454 16593 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:57.365347 16665 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:20:57.302310 23811 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:20:57.385306 24000 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:20:57.405829 23694 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:20:57.411621 24002 log.cc:927] Time spent T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.240s	user 0.003s	sys 0.000s
I20260709 13:20:57.420873 24005 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:57.421567 24005 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:57.403659 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:20:57.439891 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46048.7us
I20260709 13:20:57.453451 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:57.455888 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 52395.5us
W20260709 13:20:57.457712 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 97022.4us
I20260709 13:20:57.462635 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:20:57.465521 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:57.465758 17414 raft_consensus.cc:3060] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:57.465849 23299 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:20:57.451718 23923 raft_consensus.cc:993] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:57.472137 16743 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:20:57.469995 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:57.442296 23788 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:57.478497 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 336) took 9586 ms. Trace:
I20260709 13:20:57.460273 23621 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:20:57.484592 16745 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:20:57.478175 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:57.487226 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:57.487848 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:57.488469 16740 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:20:57.488065 16741 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:20:57.473977 23970 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:20:57.479564 23873 raft_consensus.cc:1081] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:20:57.466261 23940 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:20:57.490477 23912 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:20:57.481904 16489 rpcz_store.cc:276] 0709 13:20:47.878820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:47.879167 (+   347us) service_pool.cc:224] Handling call
0709 13:20:48.686235 (+807068us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:20:57.465259 (+8779024us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:54.391790 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:54.392695 (+   905us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:54.392753 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:20:54.392780 (+    27us) tablet.cc:662] Decoding operations
0709 13:20:54.393782 (+  1002us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:54.420858 (+ 27076us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:54.443584 (+ 22726us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:20:54.466679 (+ 23095us) tablet.cc:701] Row locks acquired
0709 13:20:54.466746 (+    67us) write_op.cc:260] PREPARE: finished
0709 13:20:54.467011 (+   265us) write_op.cc:270] Start()
0709 13:20:54.467220 (+   209us) write_op.cc:276] Timestamp: P: 1783603254466930 usec, L: 0
0709 13:20:54.467253 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:20:54.468053 (+   800us) log.cc:844] Serialized 3191 byte log entry
0709 13:20:55.112994 (+644941us) op_driver.cc:464] REPLICATION: finished
0709 13:20:55.364270 (+251276us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327cada0ac 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f328361aef0
0709 13:20:55.593404 (+229134us) write_op.cc:301] APPLY: starting
0709 13:20:55.607792 (+ 14388us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:55.921369 (+313577us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:55.921396 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:56.060658 (+139262us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:56.060899 (+   241us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:20:56.060947 (+    48us) write_op.cc:312] APPLY: finished
0709 13:20:56.061446 (+   499us) log.cc:844] Serialized 81 byte log entry
0709 13:20:56.247742 (+186296us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:56.255518 (+  7776us) write_op.cc:454] Released schema lock
0709 13:20:56.258489 (+  2971us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:56.276966 (+ 18477us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:56.304218 (+ 27252us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2511879,"reported_tablets":15,"spinlock_wait_cycles":7649536,"child_traces":[["op",{"apply.queue_time_us":455489,"apply.run_cpu_time_us":31682,"apply.run_wall_time_us":682878,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":249255,"prepare.run_cpu_time_us":5223,"prepare.run_wall_time_us":76609,"replication_time_us":604752,"spinlock_wait_cycles":26292096,"thread_start_us":592491,"threads_started":3,"wal-append.queue_time_us":282281,"wal-append.run_cpu_time_us":1012,"wal-append.run_wall_time_us":1006408}]]}
I20260709 13:20:57.437036 24005 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:57.496657 23811 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:20:57.520370 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:57.488958 23788 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:57.515548 23595 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:20:57.537335 16737 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:20:57.530941 24005 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:57.566183 16589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:20:57.566829 16589 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:20:57.567174 16589 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:57.529974 23912 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:57.543332 23951 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:57.632797 16559 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.15.29.62:37247, user_credentials={real_user=slave}} blocked reactor thread for 63923us
I20260709 13:20:57.562548 23925 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:57.568660 24021 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:57.677173 17414 raft_consensus.cc:2468] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:57.679374 23811 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:57.665498 24017 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:20:57.715294 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:57.696045 24021 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:57.668948 23951 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.680558 23925 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:57.774473 23925 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:57.775200 24018 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:20:57.777109 24016 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 13:20:57.761365 24021 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:20:57.794601 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 311135us
I20260709 13:20:57.753998 23811 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:57.712536 23912 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.172910 23811 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.150899 23788 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:20:58.237421 23942 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.884s	user 0.007s	sys 0.005s
W20260709 13:20:58.241428 24005 consensus_meta.cc:287] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.674s	user 0.004s	sys 0.006s
I20260709 13:20:58.249336 24021 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.251170 24021 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:58.264675 24015 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.855s	user 0.004s	sys 0.000s
I20260709 13:20:58.266026 17415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 2117) took 1157 ms. Trace:
I20260709 13:20:58.266464 17415 rpcz_store.cc:276] 0709 13:20:57.108553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.108717 (+   164us) service_pool.cc:224] Handling call
0709 13:20:57.109033 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:57.114488 (+  5455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.114518 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:57.122202 (+  7684us) log.cc:844] Serialized 26 byte log entry
0709 13:20:57.196701 (+ 74499us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b18003a5760. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:20:57.363942 (+167241us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:57.363984 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.365000 (+  1016us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:58.263924 (+898924us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:58.265760 (+  1836us) raft_consensus.cc:1685] finished
0709 13:20:58.265784 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:58.265954 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1718,"raft.queue_time_us":5550,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":5858,"spinlock_wait_cycles":76912256,"thread_start_us":167054,"threads_started":1,"wal-append.queue_time_us":284563}
I20260709 13:20:58.275520 24005 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.276304 24005 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:58.277298 16665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.008s	sys 0.004s
W20260709 13:20:58.277705 16665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.009s	sys 0.004s
W20260709 13:20:58.275861 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.282850 24005 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.290733 24028 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.291417 24028 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.292510 24026 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.293114 24026 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.294260 24028 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.295689 24026 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.310580 23948 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.311290 23948 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.314525 23809 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.315140 23948 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.315196 23809 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.317628 23809 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.328110 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:58.329156 23946 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:20:58.337347 24012 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.337920 24012 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.339845 24012 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:58.345011 24029 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.525s	user 0.003s	sys 0.000s
I20260709 13:20:58.377947 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:20:58.392591 24006 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.393272 24006 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.395661 24006 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:58.410296 16637 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.022s	sys 0.023s
W20260709 13:20:58.419073 16637 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.024s	sys 0.024s
W20260709 13:20:58.411583 16634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.005s	sys 0.006s
W20260709 13:20:58.420841 16634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.006s	sys 0.007s
I20260709 13:20:58.406622 23869 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:58.428486 23869 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:58.430001 23869 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.435272 24034 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.436233 24034 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.441363 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:58.441641 24034 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.442198 16587 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:58.441363 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:58.402559 16487 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 16 to 17, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 17 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:20:58.442492 24034 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.453445 24034 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.460776 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.467104 16588 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:58.485916 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.488766 24034 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.496833 24034 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.497323 24034 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.495715 24006 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:20:58.494208 16666 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:20:58.493920 24027 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.502938 24027 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.503279 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.504081 16668 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:58.506192 24027 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.506906 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.512096 16670 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:20:58.513831 24027 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:20:58.519867 24034 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.528827 24027 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.529417 24027 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.530107 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.533372 24027 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.536386 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:58.555435 24032 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.385s	user 0.002s	sys 0.000s
I20260709 13:20:58.572453 23335 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.573251 23335 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.579526 23335 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.583696 23931 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.584301 23931 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.586345 23931 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.586915 23335 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.587477 23335 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.589192 23906 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.589774 23906 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.590281 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.590006 23335 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.591902 23906 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.592329 23906 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.592852 23906 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.593793 23931 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:20:58.595942 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.594784 23906 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.644558 16670 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:58.647936 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:58.648689 17414 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:58.639099 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.657245 23335 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.657191 24027 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.658330 23335 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.657137 24034 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.668419 24034 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:58.671331 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.676265 24012 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:20:58.677255 16589 consensus_meta.cc:287] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.108s	user 0.017s	sys 0.007s
I20260709 13:20:58.677139 24012 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.677834 16589 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.679410 16590 raft_consensus.cc:1240] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: []
I20260709 13:20:58.682703 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.683351 24026 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.683902 24026 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.659585 24027 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.686105 24026 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.688285 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:58.689095 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.689723 16743 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:20:58.690528 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 99736.9us
I20260709 13:20:58.692135 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.693568 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.696429 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.697073 16668 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:20:58.696152 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.698144 16744 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:58.698542 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.660904 23809 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.694734 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.658746 23906 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.701468 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.702308 23906 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.702697 24041 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:20:58.703049 16671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.001s	user 0.018s	sys 0.003s
I20260709 13:20:58.703334 24034 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:20:58.703588 16671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.018s	sys 0.003s
I20260709 13:20:58.704316 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.704825 16740 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:20:58.706736 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:58.706955 16589 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.709591 23335 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.714287 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3818) took 1141 ms. Trace:
I20260709 13:20:58.715335 16589 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38576 (request call id 5066) took 1146 ms. Trace:
I20260709 13:20:58.715812 16589 rpcz_store.cc:276] 0709 13:20:57.561685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.561848 (+   163us) service_pool.cc:224] Handling call
0709 13:20:58.707484 (+1145636us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:58.720789 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:20:58.698999 23933 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.715631 16671 rpcz_store.cc:276] 0709 13:20:57.564593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.564746 (+   153us) service_pool.cc:224] Handling call
0709 13:20:57.565066 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:57.569656 (+  4590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.569686 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:57.667424 (+ 97738us) log.cc:844] Serialized 26 byte log entry
0709 13:20:58.554820 (+887396us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:20:58.704144 (+149324us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:58.704205 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:58.704309 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:58.704370 (+    61us) raft_consensus.cc:1685] finished
0709 13:20:58.704388 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:58.706012 (+  1624us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3905,"raft.queue_time_us":161,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5534,"spinlock_wait_cycles":57805184,"thread_start_us":1002490,"threads_started":1,"wal-append.queue_time_us":474336}
I20260709 13:20:58.723277 24048 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.723964 24048 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.724436 24008 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:58.725070 24008 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.725602 24008 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:58.726492 24048 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.706054 24006 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.728442 23906 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.730185 24049 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.703154 24041 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:20:58.740964 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46780.7us
I20260709 13:20:58.742271 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.740069 24006 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.738837 23933 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.760440 24026 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:20:58.707378 23809 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.740912 24041 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:58.762238 16743 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:58.763247 23809 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.759258 24012 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.764324 23809 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:58.764439 24012 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.764750 23809 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.765175 23809 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:58.765031 24012 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.766355 23933 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.766551 24006 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.766877 23933 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.767429 23933 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.769438 23933 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.761441 24026 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:20:58.770228 24026 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:58.772214 24041 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:58.773147 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.773867 24008 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.778841 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.779649 16664 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 13:20:58.781616 24030 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.612s	user 0.003s	sys 0.003s
I20260709 13:20:58.782575 23809 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.783692 16664 raft_consensus.cc:1240] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
I20260709 13:20:58.784756 23809 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.785756 24008 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.786002 24026 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.786777 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.788190 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:58.788751 16666 raft_consensus.cc:3055] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:20:58.789040 16666 raft_consensus.cc:740] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:58.790112 16666 consensus_queue.cc:260] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.791266 16666 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:20:58.791326 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.794077 24012 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.794523 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:58.795893 16671 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:20:58.761518 17376 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.784s (SENT) [suppressed 6 similar messages]
I20260709 13:20:58.742241 24027 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:58.740087 23960 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.803335 23960 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.806689 23960 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.743770 24049 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.810235 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:58.811082 17410 raft_consensus.cc:2393] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:20:58.813030 23951 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.817469 23694 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.818192 23694 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.819816 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.821710 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.822383 16738 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:20:58.823951 24041 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.824033 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:58.824909 17410 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
W20260709 13:20:58.824942 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 63452us
I20260709 13:20:58.826401 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:20:58.827399 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 66784.6us
I20260709 13:20:58.828087 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.819098 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:58.819489 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.833271 17416 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:20:58.833717 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:20:58.834349 16743 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:20:58.835603 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:20:58.836910 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:20:58.838310 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.838660 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:58.840029 16664 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:20:58.837400 17410 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:20:58.840816 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:58.841696 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:58.841771 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.841924 16666 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:58.830631 24055 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:58.841212 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:58.847468 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:58.847527 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:58.848209 16596 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:58.836608 23809 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:20:58.852270 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:58.849823 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:58.855000 16667 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:20:58.858299 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:58.857333 23809 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.851058 24010 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:58.860440 23809 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:20:58.860885 23694 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.840476 23951 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.863914 23951 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.866226 23951 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:58.866333 16662 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:58.857432 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:58.859994 24012 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:58.867619 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:20:58.868278 24028 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:58.868533 23621 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:20:58.868693 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.869078 23621 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.866818 16662 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:20:58.869654 23621 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:58.867655 24012 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.871388 24012 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:58.858083 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.870257 16662 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.875702 16596 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:20:58.844851 24054 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:20:58.868713 24028 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.898204 24028 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:20:58.895627 24008 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:58.901127 24008 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:58.895949 23906 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.848853 24055 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.879822 24054 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.855480 24059 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.882155 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:20:58.888466 23951 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:58.880302 24049 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.884426 24058 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.895253 16662 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:20:58.961221 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79156.6us
I20260709 13:20:59.075897 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:20:59.191200 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:59.216884 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:20:59.419929 16634 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.007s	sys 0.000s
W20260709 13:20:59.559772 24028 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.661s	user 0.007s	sys 0.004s
I20260709 13:20:59.655807 24028 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:59.539014 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 319274us
I20260709 13:20:59.658610 24028 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.660461 24028 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:59.660769 24028 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:59.661089 24028 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.664752 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:20:59.655386 16634 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.010s	sys 0.001s
I20260709 13:20:59.671837 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:59.671967 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:59.673345 16589 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 16.
I20260709 13:20:58.956869 23933 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:20:59.199033 24041 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:20:58.926683 24055 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.710310 24055 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.711527 24055 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:58.931010 24027 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:59.714829 24027 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.718364 23869 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:20:59.720556 23980 consensus_queue.cc:1059] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:20:59.721788 23980 raft_consensus.cc:3055] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:20:59.722121 23980 raft_consensus.cc:740] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:59.174816 24049 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.729522 24049 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:59.559949 23972 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.009s
I20260709 13:20:59.076337 24050 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.734014 24050 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.736519 24050 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:58.941213 23694 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:59.737361 23694 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.739377 23503 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:59.739939 23503 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.740355 23694 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.200824 24010 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:20:59.214325 23951 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:59.742229 23951 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:59.453919 16766 debug-util.cc:398] Leaking SignalData structure 0x7b0800393ee0 after lost signal to thread 15479
I20260709 13:20:58.938094 24054 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:20:59.751880 23972 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.003s	sys 0.010s
I20260709 13:20:59.232116 24012 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.754470 24012 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.669512 24026 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.758447 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1588) took 1371 ms. Trace:
I20260709 13:20:59.758934 17420 rpcz_store.cc:276] 0709 13:20:58.386888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.757525 (+1370637us) service_pool.cc:224] Handling call
0709 13:20:59.757853 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.758039 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.758062 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:59.758137 (+    75us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:20:59.758197 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.758276 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:59.758387 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:20:59.187568 24061 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:59.760782 24061 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.637892 24052 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:59.765184 24052 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.697880 24069 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.766330 24069 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.766737 24000 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:59.767290 24000 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.768486 24069 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.666451 23919 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.769392 24000 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:59.769237 23919 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.771521 23919 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.701020 16662 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:59.701861 24073 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.772703 24073 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.707906 23969 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:59.775281 23969 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:20:59.777148 24040 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
W20260709 13:20:59.777892 16662 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 3359) took 2258 ms (client timeout 1778 ms). Trace:
W20260709 13:20:59.778419 16662 rpcz_store.cc:269] 0709 13:20:57.519800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.520012 (+   212us) service_pool.cc:224] Handling call
0709 13:20:58.866199 (+1346187us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400138920. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb5bdc0 00007f327fbe7e18 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6
0709 13:20:59.777778 (+911579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3087769472}
I20260709 13:20:59.779731 24073 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.781646 23969 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.708725 24063 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.720708 24060 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:20:59.765491 16665 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38524 (request call id 1258) took 1233 ms. Trace:
I20260709 13:20:59.783803 16665 rpcz_store.cc:276] 0709 13:20:58.532515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.532680 (+   165us) service_pool.cc:224] Handling call
0709 13:20:58.540443 (+  7763us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:58.540791 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:58.540808 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:58.541147 (+   339us) log.cc:844] Serialized 26 byte log entry
0709 13:20:59.764763 (+1223616us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b54000ce2a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fb5b6b8 00007f327fbe492b 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c
0709 13:20:59.764852 (+    89us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:59.764925 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.765024 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:20:59.765089 (+    65us) raft_consensus.cc:1685] finished
0709 13:20:59.765110 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:59.765409 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":10352,"spinlock_wait_cycles":3554176,"thread_start_us":11493,"threads_started":1,"wal-append.queue_time_us":9243,"wal-append.run_cpu_time_us":1918,"wal-append.run_wall_time_us":1015148}
I20260709 13:20:59.786193 24073 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:20:59.786729 24073 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:59.787856 24073 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.789459 24063 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.790417 23980 consensus_queue.cc:260] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:59.794266 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:20:58.911381 23906 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.430219 23960 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:58.941268 24021 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:58.941334 24056 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:20:59.799026 16708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.016s	sys 0.077s
I20260709 13:20:58.902796 23621 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.200227 23809 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:59.811414 16708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.016s	sys 0.077s
I20260709 13:20:59.687134 23933 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.694572 24041 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:20:58.941902 24059 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.810663 24027 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:59.836088 16738 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 19: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:20:59.827029 23933 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:20:59.176404 24058 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.832881 24067 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.829491 23503 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.831208 24041 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:20:59.842772 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.810181 23960 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.845180 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.845865 16669 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:20:59.836594 24061 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.824002 24056 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:20:59.876266 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 183074us
W20260709 13:20:59.881415 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 58791.6us
I20260709 13:20:59.834355 24021 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.882019 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:59.845952 23906 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:59.847570 24063 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.887434 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:59.887581 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.890367 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.892023 23809 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:59.879628 24012 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.895182 24012 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.895494 24052 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.897260 24012 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.897475 24056 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.899091 24058 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.899863 24055 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.900122 24059 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.900453 24071 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.900851 23621 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.900962 24071 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.846148 24067 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.903496 24028 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.904884 24067 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.889376 24061 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:20:59.905588 24028 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.905764 24061 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:20:59.885599 16770 heartbeater.cc:646] Failed to heartbeat to 127.15.29.62:37247 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.15.29.62:37247 timed out after 15.000s (SENT)
I20260709 13:20:59.879914 24027 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.879992 24068 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.906793 24061 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.879652 23503 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:20:59.915952 23933 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.881219 23969 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:59.926258 24052 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:20:59.926497 24059 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:20:59.926892 24052 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.927386 24059 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:59.928392 24059 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.929186 24052 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.918434 24027 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.926486 23969 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.931350 24027 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.931867 23969 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.932214 24071 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.934078 23933 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:59.934559 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47803.4us
I20260709 13:20:59.926566 23906 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:20:59.935783 23906 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:20:59.936264 23906 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:20:59.936777 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:59.937454 23906 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.937731 24054 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.938735 23906 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:20:59.939615 24053 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:20:59.940088 24053 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:20:59.940492 24053 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:20:59.942198 24028 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.917376 23960 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.942957 23621 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.942797 24028 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.943436 24068 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:20:59.917768 24021 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.944185 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.944814 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.945391 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.945894 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.945994 16667 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:20:59.946857 16668 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:20:59.947041 24026 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:20:59.947188 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.947602 24026 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.948874 16669 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:20:59.949745 24028 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.947525 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.943549 23621 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.946758 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.955052 16664 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:20:59.945978 24068 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:20:59.958981 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:59.959913 16589 raft_consensus.cc:2410] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:20:59.960835 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:59.965521 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:20:59.966090 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4489) took 1285 ms. Trace:
I20260709 13:20:59.966385 16593 raft_consensus.cc:2410] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:20:59.967159 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:20:59.957087 23503 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:20:59.951746 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.966527 16741 rpcz_store.cc:276] 0709 13:20:58.680672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.965342 (+1284670us) service_pool.cc:224] Handling call
0709 13:20:59.965625 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.965799 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.965824 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:59.965862 (+    38us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:20:59.965900 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:59.965949 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:20:59.966035 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:20:59.952612 16561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.672s	user 0.004s	sys 0.027s
I20260709 13:20:59.950073 24026 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:20:59.970172 16561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.690s	user 0.004s	sys 0.027s
W20260709 13:20:59.980656 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 2.70036e+06us
I20260709 13:20:59.981689 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:20:59.973978 23503 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:20:59.974288 23621 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:20:59.976799 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:20:59.978704 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:20:59.978612 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:20:59.983911 16664 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:20:59.993340 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:59.994084 16665 raft_consensus.cc:2393] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:20:59.996286 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:20:59.994802 24021 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:20:59.998529 24021 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:20:59.999799 24021 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.002295 24054 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:00.006143 16708 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.009093 17420 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:21:00.011188 16737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 1891) took 3060 ms (client timeout 1784 ms). Trace:
W20260709 13:21:00.011821 16737 rpcz_store.cc:269] 0709 13:20:56.950688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:56.960064 (+  9376us) service_pool.cc:224] Handling call
0709 13:21:00.010850 (+3050786us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b54003d4354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:00.011022 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1046879616}
I20260709 13:21:00.013249 24055 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:00.019162 24051 raft_consensus.cc:2804] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:00.019373 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4495) took 1327 ms. Trace:
I20260709 13:21:00.020200 16736 rpcz_store.cc:276] 0709 13:20:58.692221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.018504 (+1326283us) service_pool.cc:224] Handling call
0709 13:21:00.018809 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.019017 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.019043 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.019089 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:21:00.019136 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.019191 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.019305 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:00.027449 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.028101 16736 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:00.029036 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:00.029783 16663 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:00.038774 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62043.3us
I20260709 13:20:59.996234 24041 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.043274 16662 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:00.044263 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.045655 24041 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:00.047102 23811 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:00.047562 23811 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.047930 23811 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:00.055277 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:00.045979 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:00.020094 24051 raft_consensus.cc:697] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:00.061594 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.065196 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.065874 16739 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 11.
I20260709 13:20:59.996744 24068 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:00.054746 24055 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:21:00.060511 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.072841 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.078116 16739 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:00.080376 16559 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:00.081028 16663 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:00.081827 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 3364) took 1305 ms. Trace:
I20260709 13:21:00.083429 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:00.082167 16663 rpcz_store.cc:276] 0709 13:20:58.776816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.028736 (+1251920us) service_pool.cc:224] Handling call
0709 13:21:00.081774 (+ 53038us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:00.085762 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.086614 17410 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:21:00.089980 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:00.090302 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:00.091804 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:00.097389 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.097827 24051 consensus_queue.cc:237] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.075098 23595 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:00.100052 23595 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.084072 24052 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 13:21:00.101126 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1583) took 1715 ms. Trace:
I20260709 13:21:00.101598 17412 rpcz_store.cc:276] 0709 13:20:58.385365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.088671 (+1703306us) service_pool.cc:224] Handling call
0709 13:21:00.100477 (+ 11806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.100714 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.100745 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.100791 (+    46us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:00.100852 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.100911 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.101039 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:00.101605 17410 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:00.107164 24075 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:00.107758 24075 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:00.108177 24075 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:00.112039 17410 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:00.111735 23811 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.102751 23595 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:00.115487 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.116250 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.117134 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.118783 16739 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:00.118796 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.116197 16670 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:00.120851 24053 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.121431 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.124577 23811 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:00.126276 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.127208 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.128597 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 5076) took 1702 ms. Trace:
I20260709 13:21:00.131922 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:00.133970 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.133023 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.134593 16740 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:00.134734 17420 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:00.129086 16590 rpcz_store.cc:276] 0709 13:20:58.426368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.427741 (+  1373us) service_pool.cc:224] Handling call
0709 13:20:58.428033 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:58.682317 (+254284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.128333 (+1446016us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540023afd4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:00.128531 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577668480}
I20260709 13:21:00.134159 16670 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:00.139235 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.140163 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.140851 17412 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:00.141215 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.141834 17416 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
W20260709 13:21:00.115998 23912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.967s	user 0.005s	sys 0.027s
I20260709 13:21:00.153504 24075 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.128847 24066 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:00.156021 24066 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:00.157192 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:00.157073 24066 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:00.157781 23912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.009s	user 0.006s	sys 0.028s
I20260709 13:21:00.157871 16663 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:00.128507 23906 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:00.160094 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.163429 23960 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:00.165632 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:00.166082 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.166273 17414 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:00.173379 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.165676 24049 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.162484 24053 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:00.175503 23906 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:00.176463 23906 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:00.177939 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.179590 24052 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:00.179777 23980 raft_consensus.cc:3060] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:00.174626 17413 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48618 (request call id 1590) took 1777 ms (client timeout 2131 ms). Trace:
W20260709 13:21:00.181128 17413 rpcz_store.cc:269] 0709 13:20:58.397102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.173046 (+1775944us) service_pool.cc:224] Handling call
0709 13:21:00.174553 (+  1507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:00.181686 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.184710 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.187119 16663 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:00.180152 24052 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.190806 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.191349 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.192548 24052 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.192649 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.192869 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.193161 24025 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:00.193794 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.193728 24025 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.189359 23489 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:21:00.195386 16738 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37644 (request call id 4504) took 1363 ms (client timeout 1721 ms). Trace:
I20260709 13:21:00.190409 16743 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 13:21:00.195833 16738 rpcz_store.cc:269] 0709 13:20:58.832010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.835376 (+  3366us) service_pool.cc:224] Handling call
0709 13:21:00.195189 (+1359813us) spinlock_profiling.cc:243] Waited 997 ms on lock 0x7b540007c6d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:00.195310 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2293323008}
I20260709 13:21:00.191447 16741 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:00.195169 23489 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.187302 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.205461 16740 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:00.190655 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.197548 24008 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:00.200980 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.206936 16559 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:00.201699 16670 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:00.203080 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.204327 17413 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:21:00.207252 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.207821 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.212361 17414 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:21:00.214521 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.215698 16739 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:00.217475 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.216619 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.221961 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.222021 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.223242 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.226024 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.226555 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.224882 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.227293 16589 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:00.224478 16587 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 11.
I20260709 13:21:00.224543 24051 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:00.226076 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.228751 24051 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.229362 16588 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 10.
I20260709 13:21:00.224083 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.232123 16587 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:00.234016 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.236150 24040 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:00.236991 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.236797 24040 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.237653 16737 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:00.234782 16744 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:00.239287 24040 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.240032 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.240278 16740 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:00.240732 16744 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:00.241932 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.224579 23811 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:00.242883 16665 raft_consensus.cc:1240] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:00.244062 16559 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:00.244228 24061 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:00.243455 16590 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:00.244359 24005 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:00.245915 24031 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:00.246381 24005 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:00.242676 16671 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:00.247471 24005 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.248695 16669 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:00.244077 23869 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:00.249750 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.250097 23869 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.244987 24061 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.266165 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.251885 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.253294 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.252682 16662 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:00.254081 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.195847 24025 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.294965 16591 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 5070) took 1870 ms. Trace:
I20260709 13:21:00.258391 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.258586 17414 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:21:00.258283 16745 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:00.260321 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:00.261538 16737 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 14.
I20260709 13:21:00.246534 24031 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:00.266379 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.241389 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.236506 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.208078 24021 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:21:00.267459 16587 raft_consensus.cc:2468] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:00.371220 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:00.371417 24031 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.374238 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.375363 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.376389 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.376933 16737 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:00.377722 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.371873 16670 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:00.325349 24054 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.382700 16740 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:00.383607 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:00.322055 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.390200 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:00.373088 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.391760 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:00.391795 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:00.394116 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:00.298848 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53605.6us
W20260709 13:21:00.329828 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.379156 16588 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:00.314013 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.295590 16591 rpcz_store.cc:276] 0709 13:20:58.424714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:59.710298 (+1285584us) service_pool.cc:224] Handling call
0709 13:20:59.710629 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.765392 (+ 54763us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.765423 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:59.765466 (+    43us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:20:59.765510 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.294678 (+529168us) spinlock_profiling.cc:243] Waited 54.5 ms on lock 0x7b54003395d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:00.294739 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.294896 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125428480}
I20260709 13:21:00.299366 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.398888 16662 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:21:00.395689 16744 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
I20260709 13:21:00.398398 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1586) took 2012 ms. Trace:
I20260709 13:21:00.402099 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.379390 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.403249 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.371883 17419 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:00.407711 16591 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:21:00.407243 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.379549 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:00.365074 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.402194 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.380647 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.405195 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.397671 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.400820 17411 rpcz_store.cc:276] 0709 13:20:58.386261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.397560 (+2011299us) service_pool.cc:224] Handling call
0709 13:21:00.397838 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:00.398021 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.398046 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:00.398089 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:00.398134 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:00.398199 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.398331 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:21:00.362483 23931 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.416759 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:00.419826 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.417973 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.419857 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.421007 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:00.421720 16590 raft_consensus.cc:2393] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:00.421267 16591 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:00.422581 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.423497 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.422605 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.424964 16591 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:21:00.425215 16662 raft_consensus.cc:2393] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 21: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:00.381193 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.425765 17415 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:00.426112 16591 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:00.311484 24051 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.428036 16739 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 20: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:00.428530 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:00.298650 24075 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:00.243479 23811 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.322636 24000 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:00.431243 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.437664 16741 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:21:00.321031 23960 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:00.394443 24072 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:00.446396 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.448849 16741 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:00.326022 23972 consensus_queue.cc:1059] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.119s
I20260709 13:21:00.416463 24061 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.315265 24025 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.120s
I20260709 13:21:00.450098 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:00.393491 24054 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.464740 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.452924 23960 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.470813 24073 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:00.471962 23960 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:00.473158 23811 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.453114 24000 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.474982 16663 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:00.432706 23931 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.475648 24000 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:00.476809 23811 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:00.477376 23931 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.478408 17419 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:00.479158 24061 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:00.479554 24061 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:00.480391 24061 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.481581 24061 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:00.482897 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.483572 16738 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:00.483772 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:21:00.484719 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:00.485046 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88311.8us
I20260709 13:21:00.450729 23489 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.495405 16593 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:21:00.495939 16737 raft_consensus.cc:2376] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17. Re-sending same reply.
I20260709 13:21:00.498150 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.507436 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.510040 23869 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:00.508944 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:00.455235 24072 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.500924 23918 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:21:00.524049 23918 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:00.524441 23918 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:00.525202 23918 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.526401 23918 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:00.528003 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.529239 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:00.530370 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.508486 23906 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:00.535215 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:00.537599 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:00.539628 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:00.542956 24072 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:00.549619 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.551229 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.552628 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.554000 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.555291 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.555455 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.556169 16592 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:00.556927 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.557159 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.557756 16589 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:21:00.559257 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.559628 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.560707 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.561218 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.561437 16590 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:00.562906 16589 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:00.596386 24041 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:00.500097 23960 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
W20260709 13:21:00.597664 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58105us
I20260709 13:21:00.532563 23595 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:00.601604 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.602555 23489 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:00.631317 16667 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:00.636504 24056 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:00.636148 23489 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:00.637128 24056 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:00.536216 23906 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:00.558745 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:21:00.683487 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 82004.3us
I20260709 13:21:00.684660 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.685966 23636 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:00.686486 23636 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:00.686901 23636 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:00.687489 24014 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 3.279s	user 0.005s	sys 0.033s
W20260709 13:21:00.707059 17376 leader_election.cc:336] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.784s (SENT)
I20260709 13:21:00.724131 17376 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:00.628917 23595 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:00.638827 24071 raft_consensus.cc:2764] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:21:00.674032 23906 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:00.751609 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:00.777048 16595 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47022 (request call id 1451) took 3511 ms. Trace:
I20260709 13:21:00.784933 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:00.793170 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:00.831426 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 774086us
I20260709 13:21:00.777729 16595 rpcz_store.cc:276] 0709 13:20:57.231379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.235490 (+  4111us) service_pool.cc:224] Handling call
0709 13:20:57.251256 (+ 15766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:20:57.251702 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:57.251742 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:20:57.252399 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:20:57.371897 (+119498us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:20:57.371958 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:20:57.372058 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:00.686916 (+3314858us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:21:00.688960 (+  2044us) raft_consensus.cc:1685] finished
0709 13:21:00.688982 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:00.743074 (+ 54092us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1409,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"spinlock_wait_cycles":53775616,"thread_start_us":118978,"threads_started":1,"wal-append.queue_time_us":155323}
W20260709 13:21:00.845409 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 160874us
I20260709 13:21:00.847266 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.667822 23869 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.113s
I20260709 13:21:00.746423 23595 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:00.743994 23960 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:21:00.890751 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 43634.5us
I20260709 13:21:00.906971 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:00.908110 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:00.908335 24091 log.cc:927] Time spent T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.372s	user 0.000s	sys 0.000s
W20260709 13:21:00.900945 16587 consensus_meta.cc:287] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:21:00.908954 16587 raft_consensus.cc:2468] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
W20260709 13:21:00.911895 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003973a0 after lost signal to thread 16528
W20260709 13:21:00.912398 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 206032us
W20260709 13:21:00.913264 16766 debug-util.cc:398] Leaking SignalData structure 0x7b080024ada0 after lost signal to thread 16620
W20260709 13:21:00.914391 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003af4c0 after lost signal to thread 16695
W20260709 13:21:00.914654 16766 debug-util.cc:398] Leaking SignalData structure 0x7b0800342220 after lost signal to thread 16696
I20260709 13:21:00.916509 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:00.917181 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:00.917395 16590 raft_consensus.cc:2410] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:21:00.920184 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08004111e0 after lost signal to thread 16769
I20260709 13:21:00.921778 16662 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:00.922118 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:21:00.922348 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08002cb440 after lost signal to thread 17446
W20260709 13:21:00.986145 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003610a0 after lost signal to thread 17447
I20260709 13:21:00.987212 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:00.923422 16663 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:21:01.016845 16739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.001s	sys 0.014s
W20260709 13:21:01.017298 16739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.001s	sys 0.014s
I20260709 13:21:01.019475 16745 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:00.934326 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.890508 24053 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:01.022300 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:01.022939 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 103316us
I20260709 13:21:01.023237 16742 raft_consensus.cc:2410] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:21:00.891297 24041 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:21:01.024710 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:00.924444 23941 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:01.035316 23941 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.035871 23941 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:01.036693 24092 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.503s	user 0.000s	sys 0.001s
I20260709 13:21:01.022397 23946 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:01.044548 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:01.047353 24056 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:01.033300 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.048648 24040 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.061642 24041 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.062758 24041 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.074096 24054 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.065086 23941 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:01.074934 16741 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.046s	sys 0.068s
W20260709 13:21:01.075644 16741 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.047s	sys 0.068s
W20260709 13:21:01.035923 24095 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.185s	user 0.000s	sys 0.000s
W20260709 13:21:01.077824 16736 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.017s	sys 0.024s
I20260709 13:20:59.483345 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:21:01.078318 16736 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.017s	sys 0.024s
I20260709 13:21:00.918643 24008 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:01.090474 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:21:01.075157 24031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.015s	sys 0.042s
I20260709 13:21:01.097337 23980 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:01.097616 16595 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:01.097888 23980 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:01.098942 23980 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.101191 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.104374 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.104746 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.103291 23941 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.107632 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:21:01.108749 17374 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.856s (SENT)
I20260709 13:21:01.108081 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.110481 16588 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:01.112247 16596 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 5091) took 2227 ms. Trace:
W20260709 13:21:01.095034 24031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.015s	sys 0.042s
I20260709 13:21:01.115101 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.115990 16742 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:01.112707 16596 rpcz_store.cc:276] 0709 13:20:58.884890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:58.885175 (+   285us) service_pool.cc:224] Handling call
0709 13:20:58.885598 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:20:59.847199 (+961601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:20:59.847231 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:20:59.847277 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:20:59.847322 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.111970 (+1264648us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b5400244854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:01.112034 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:01.112188 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2211040128}
I20260709 13:21:01.116387 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:01.123653 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.133543 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.134267 17410 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:21:01.135965 16739 raft_consensus.cc:2393] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:01.138362 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.139710 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.143662 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.144934 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.145500 16740 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:01.146234 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.147955 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.148073 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.148947 17420 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 20: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:01.149348 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.150056 16737 raft_consensus.cc:2393] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:01.141219 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.153730 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.153975 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.155664 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.156231 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.160542 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.177562 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.154829 23948 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:21:01.162712 23960 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.183297 16737 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:21:01.184966 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:01.185829 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.183302 24099 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:21:01.189647 16561 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.190200 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.195719 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:21:01.195305 23636 consensus_meta.cc:287] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.508s	user 0.014s	sys 0.001s
I20260709 13:21:01.197712 23636 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.196410 23595 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:01.202014 24097 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:21:01.203176 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2803) took 1115 ms. Trace:
I20260709 13:21:01.203568 16743 rpcz_store.cc:276] 0709 13:21:00.087725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.087839 (+   114us) service_pool.cc:224] Handling call
0709 13:21:00.088157 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:00.196116 (+107959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.202824 (+1006708us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400434254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:01.203079 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234567040}
I20260709 13:21:01.180202 24000 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:01.213428 23927 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:01.214051 23927 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:01.209935 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.226625 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.202333 23636 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.229725 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.231125 23927 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.177757 23906 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.233103 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.234062 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.235571 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.236304 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.245230 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.247560 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.250600 24073 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:01.251147 24073 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.251608 24073 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:01.262238 24051 raft_consensus.cc:3055] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:01.262768 24051 raft_consensus.cc:740] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:01.245246 16710 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.267334 16738 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:01.219765 24060 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:01.270756 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.273963 16739 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 20: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:01.237241 23694 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:01.265228 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 220773us
I20260709 13:21:01.243896 23869 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.243708 24058 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:01.267299 24055 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:01.275771 24053 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.318334 23595 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:01.323009 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.327389 24100 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:21:01.328586 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1911) took 1011 ms. Trace:
I20260709 13:21:01.280974 24058 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:01.328965 16741 rpcz_store.cc:276] 0709 13:21:00.316743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.492539 (+175796us) service_pool.cc:224] Handling call
0709 13:21:00.492907 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:00.493307 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:00.493364 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:00.493966 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 13:21:01.076102 (+582136us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:01.076178 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.076283 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:01.328367 (+252084us) raft_consensus.cc:1685] finished
0709 13:21:01.328386 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:01.328529 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1973,"raft.queue_time_us":97,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5212,"thread_start_us":559355,"threads_started":1,"wal-append.queue_time_us":635396}
I20260709 13:21:01.333418 16742 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:01.278123 23694 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.263617 24051 consensus_queue.cc:260] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:01.314535 17376 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.856s (SENT)
I20260709 13:21:01.359381 17376 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:01.360657 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46151.2us
I20260709 13:21:01.280233 23927 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.343353 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.264892 24005 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:01.367224 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.367533 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.368289 16663 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:21:01.369052 24019 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:01.370694 16667 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:01.372576 16670 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:01.372387 16664 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:01.372455 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.376248 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.402108 16671 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:01.314739 24060 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:01.388890 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56204.6us
I20260709 13:21:01.359133 23694 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:21:01.393683 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 212663us
I20260709 13:21:01.395728 16738 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1909) took 1079 ms. Trace:
I20260709 13:21:01.315555 24055 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:01.410157 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:01.399817 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 239369us
I20260709 13:21:01.315553 23972 raft_consensus.cc:2764] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:21:01.380074 16666 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:21:01.411230 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.404450 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 156943us
W20260709 13:21:01.407804 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 135347us
I20260709 13:21:01.408417 16738 rpcz_store.cc:276] 0709 13:21:00.316203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:00.489796 (+173593us) service_pool.cc:224] Handling call
0709 13:21:00.504340 (+ 14544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:01.267975 (+763635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:01.395491 (+127516us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b5400426654. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:01.395675 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754230528}
I20260709 13:21:01.412722 16663 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:21:01.393497 23906 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:01.407197 23918 consensus_meta.cc:287] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.880s	user 0.011s	sys 0.009s
I20260709 13:21:01.409107 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.399262 23946 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:01.419349 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:01.423112 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.424506 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.426069 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.427100 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.430945 16492 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.432678 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.434396 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.404460 24060 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.430006 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:01.412330 24005 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:21:01.465440 23918 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:01.438313 24077 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:01.432859 16492 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 17, leader changed from <none> to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 17 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:01.468335 24005 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:01.475353 24077 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.477046 24077 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.421823 23946 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:01.480675 23946 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:01.416918 24051 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:01.419567 23489 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.418193 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:01.495990 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 77863.4us
I20260709 13:21:01.497128 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.497802 23811 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.498521 23811 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.439711 16741 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 20: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:01.505095 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:21:01.505476 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.438943 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:01.482053 24058 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.408269 24031 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:21:01.417922 23869 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:21:01.507187 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.517946 16663 raft_consensus.cc:1240] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:01.508692 16662 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:01.510004 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.510809 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:01.466315 24060 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:01.503749 16492 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:01.467155 24072 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:21:01.541249 24072 consensus_queue.cc:1243] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:01.541862 24072 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:01.542220 24072 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:01.543108 24072 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.544413 24072 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:01.429483 24041 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.520164 16668 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:01.520699 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:01.500490 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:01.522053 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 92139.2us
I20260709 13:21:01.467409 24054 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:21:01.561508 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 61080.2us
I20260709 13:21:01.561673 24054 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.562107 24054 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:01.563712 24103 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:01.564318 24103 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.565246 24103 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:01.568914 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48292.6us
I20260709 13:21:01.438762 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.526048 24102 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:01.569947 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:21:01.527089 16487 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 313) took 17471 ms (client timeout 15000 ms). Trace:
I20260709 13:21:01.570848 24006 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:01.571672 24006 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.500505 24073 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.518945 23918 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:01.572293 24031 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:01.572364 24006 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:01.572860 23918 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.572995 24031 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:01.573391 23918 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:01.573488 24031 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:01.575791 24056 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:01.521539 24005 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:21:01.413856 24111 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.108s
I20260709 13:21:01.501390 24053 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.582633 24031 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.585464 24073 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:01.547825 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.585981 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:01.587005 17376 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 2.046s (SENT)
W20260709 13:21:01.539973 24108 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:21:01.548269 23788 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:01.589079 23788 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:01.571121 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:01.514194 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 75311us
I20260709 13:21:01.590628 24031 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.591198 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:01.524081 24109 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:01.559834 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 139412us
I20260709 13:21:01.570083 24102 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.592082 24109 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.592773 24056 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:01.593374 24102 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.594367 24109 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:01.595633 23595 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:01.596065 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.596872 24056 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:01.597383 24056 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:01.596091 23595 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:01.597832 24056 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.597985 23595 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.591342 24103 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:01.592228 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.604321 24103 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:01.570557 16487 rpcz_store.cc:269] 0709 13:20:44.056190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:44.069912 (+ 13722us) service_pool.cc:224] Handling call
0709 13:21:01.526995 (+17457083us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:20:58.541888 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:20:58.542474 (+   586us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:20:58.542509 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:20:58.542528 (+    19us) tablet.cc:662] Decoding operations
0709 13:20:58.550755 (+  8227us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:20:58.550812 (+    57us) write_op.cc:641] Partition lock acquired for write op
0709 13:20:58.550844 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:20:58.551079 (+   235us) tablet.cc:701] Row locks acquired
0709 13:20:58.551114 (+    35us) write_op.cc:260] PREPARE: finished
0709 13:20:58.551315 (+   201us) write_op.cc:270] Start()
0709 13:20:58.551464 (+   149us) write_op.cc:276] Timestamp: P: 1783603258551243 usec, L: 0
0709 13:20:58.551485 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:20:58.552308 (+   823us) log.cc:844] Serialized 530 byte log entry
0709 13:20:58.583938 (+ 31630us) op_driver.cc:464] REPLICATION: finished
0709 13:20:58.692189 (+108251us) write_op.cc:301] APPLY: starting
0709 13:20:58.692270 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 13:20:58.693477 (+  1207us) tablet.cc:1370] finished BulkCheckPresence
0709 13:20:58.693568 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:20:58.694297 (+   729us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:20:58.694514 (+   217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:20:58.694556 (+    42us) write_op.cc:312] APPLY: finished
0709 13:20:58.694756 (+   200us) log.cc:844] Serialized 25 byte log entry
0709 13:20:58.705735 (+ 10979us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:58.708099 (+  2364us) write_op.cc:454] Released schema lock
0709 13:20:58.717456 (+  9357us) write_op.cc:341] FINISH: Updating metrics
0709 13:20:58.727532 (+ 10076us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:20:58.727563 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":273347,"reported_tablets":23,"spinlock_wait_cycles":13056,"child_traces":[["op",{"apply.queue_time_us":107989,"apply.run_cpu_time_us":3562,"apply.run_wall_time_us":35281,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"num_ops":1,"prepare.queue_time_us":54099,"prepare.run_cpu_time_us":6799,"prepare.run_wall_time_us":22602,"raft.queue_time_us":14055,"raft.run_cpu_time_us":32315,"raft.run_wall_time_us":113747,"replication_time_us":32294,"thread_start_us":165113,"threads_started":4,"wal-append.queue_time_us":10746,"wal-append.run_cpu_time_us":7302,"wal-append.run_wall_time_us":1145413}]]}
I20260709 13:21:01.608078 24056 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.607646 24073 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:01.612831 24118 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:01.612911 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 174248us
I20260709 13:21:01.613487 24118 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.613633 23788 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:01.596794 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.617054 24019 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:01.618182 16561 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:01.619418 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:01.620168 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.624074 24056 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.615039 16708 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.624594 24031 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:01.627303 24031 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:01.626629 16561 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:01.627738 24118 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:01.553186 23946 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.628521 24056 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.629112 16669 raft_consensus.cc:606] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.629634 16669 raft_consensus.cc:614] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:01.569025 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.627928 24031 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.630383 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.632685 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.625036 24073 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:01.634481 24073 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:01.635154 23788 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:01.630424 23946 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.636090 24056 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:01.635228 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:21:01.636374 24112 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:01.638020 24112 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.638038 16669 raft_consensus.cc:606] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.638542 24112 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:01.641877 16669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:21:01.635123 24073 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:01.645020 24056 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:01.647505 24056 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:01.651121 24109 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.651192 23636 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.651554 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.651997 23636 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.652431 24118 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.653237 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.653347 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.654011 24118 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.654395 16708 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:21:01.637094 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 68163us
I20260709 13:21:01.635776 23788 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.525368 16492 catalog_manager.cc:5697] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.625795 23918 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:01.727216 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:01.720106 16478 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 3c3ca2a0390b46bb9a5eda03a64eda1a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.683303 16587 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.656581 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.729423 16587 raft_consensus.cc:2376] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19. Re-sending same reply.
I20260709 13:21:01.656091 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.729907 16592 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:01.683655 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:01.724103 16668 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:21:01.684365 16591 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:21:01.671133 16589 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:01.665858 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.683070 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.661932 24104 raft_consensus.cc:993] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: : Instructing follower efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:21:01.654397 23636 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:01.761929 23788 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:01.664990 24054 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.694418 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.703133 24006 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.771889 23927 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:01.772408 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 106628us
I20260709 13:21:01.691049 24118 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.773039 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.799006 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.799139 16741 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:21:01.775326 16742 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:01.799827 23927 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:01.801258 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:01.800890 24118 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.803158 24073 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.804824 16589 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:01.804924 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.806361 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:01.808843 17376 leader_election.cc:336] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Timed out: RequestConsensusVote RPC to 127.15.29.3:36655 timed out after 1.611s (SENT)
I20260709 13:21:01.809523 17376 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:01.809677 23636 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:01.808924 16742 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:01.810948 24041 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:01.811439 24041 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.812075 16593 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:01.812937 23927 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:01.813401 23927 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:01.812636 24041 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.813982 23927 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.776466 23918 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:01.815613 24041 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:01.816494 24054 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.665411 24109 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 13:21:01.822057 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:01.728307 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73982us
I20260709 13:21:01.824899 23980 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:01.814370 24104 raft_consensus.cc:1081] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Signalling peer efa00eb5e7404570958fdebb6cd077e8 to start an election
I20260709 13:21:01.828001 24055 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.828650 24118 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.730252 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:01.779526 16744 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:21:01.780046 24031 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.781023 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.780632 24060 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.670128 23595 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.734411 23912 raft_consensus.cc:2804] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:01.700263 24112 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.842001 16742 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:21:01.844153 24112 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.789446 23694 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.845978 24072 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:01.846220 23694 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.732731 16492 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.730204 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:01.793304 16664 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:01.847416 24006 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.848097 24058 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:01.848959 16492 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:01.849922 24060 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.817816 16738 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:21:01.855077 24055 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:01.715683 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.819533 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.831804 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.818436 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.833076 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.835763 16741 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:01.866353 16671 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:01.868214 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:01.838143 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 107957us
I20260709 13:21:01.839735 16737 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
 from {username='slave'} at 127.0.0.1:37644
I20260709 13:21:01.829066 24103 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 13:21:01.871165 16737 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:01.871654 16737 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:01.661265 16590 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:01.853235 23595 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:01.829555 24005 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:01.881079 24005 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.884411 16592 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:01.884882 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.886354 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.886534 16708 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.886607 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.888209 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.888834 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:01.889564 16671 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 20: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:01.654721 23972 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:01.874949 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:01.877753 16669 raft_consensus.cc:606] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Received request to transfer leadership to TS efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:01.891438 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:01.878218 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:01.892284 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:01.891463 16669 raft_consensus.cc:614] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:01.879347 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:01.879678 23912 raft_consensus.cc:697] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:21:01.890991 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 72627.8us
I20260709 13:21:01.878067 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.896405 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:01.876979 16492 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.900411 16596 raft_consensus.cc:1240] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:01.899874 16492 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 18 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.901834 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:01.901554 16492 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.904381 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.906289 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.889935 16737 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.908208 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.908967 17410 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:01.837622 23918 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:01.915520 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:01.942291 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54845.6us
I20260709 13:21:01.916517 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:01.944190 17409 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:01.911597 17420 raft_consensus.cc:1275] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:01.916023 17413 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:01.954953 16737 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:01.920925 16492 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:01.908444 23490 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:01.957274 23490 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:01.956527 16492 catalog_manager.cc:5697] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 19, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:01.957672 23490 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.958154 16589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:01.958230 16492 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.890523 24104 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:21:01.959820 16492 catalog_manager.cc:5697] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 22 to 23, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 23 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:01.961355 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51391.4us
I20260709 13:21:01.962611 24104 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:01.963644 24104 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:01.963994 24104 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:01.968861 24104 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:01.961390 16492 catalog_manager.cc:5697] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:01.970232 24104 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:01.970805 16492 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:01.975891 16589 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:21:01.976383 16589 pending_rounds.cc:85] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting all ops after (but not including) 11
I20260709 13:21:01.976655 16589 pending_rounds.cc:107] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:21:01.976939 16589 raft_consensus.cc:2889] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:01.955057 16708 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:01.907527 23595 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:01.884702 24005 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.908586 24051 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:01.984527 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:01.955423 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 55558.7us
I20260709 13:21:01.899181 23694 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:01.995308 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:01.988476 23694 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.000787 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:01.955530 24054 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:21:01.988210 24102 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:21:02.009218 23694 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:02.010432 23811 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:21:02.024654 23811 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:21:02.025040 23811 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:02.026069 23811 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.027980 24110 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:02.005110 23906 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:02.029922 23906 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:02.030290 23906 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:02.032982 23912 consensus_queue.cc:237] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:01.991437 23946 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:02.040017 24056 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.042263 24056 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:01.987666 24073 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:02.043350 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.044319 17420 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.045187 23946 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:02.045711 23946 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:02.047866 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:02.052453 16666 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:21:02.053299 16666 raft_consensus.cc:606] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 LEADER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:02.011360 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:02.058771 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47499.6us
I20260709 13:21:02.059779 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:01.990037 24111 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:21:02.021390 24031 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:02.004125 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:02.088243 17420 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
W20260709 13:21:02.088815 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84757.9us
I20260709 13:21:01.999392 24055 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 13:21:02.062660 24000 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:02.083483 24019 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:02.079772 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:02.083791 24008 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:02.108548 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:02.109148 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:02.109346 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:02.110286 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:02.111430 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:02.115520 17413 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:02.116349 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.091639 23811 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:02.118422 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.112658 16710 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:02.120012 16742 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:02.120191 17414 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:02.116313 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.091607 24005 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:02.126235 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:02.128016 16665 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076"
dest_uuid: "170921ba08b74a33995ae7c7686f8109"
mode: GRACEFUL
new_leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:55430
I20260709 13:21:02.128847 16665 raft_consensus.cc:606] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Received request to transfer leadership to TS 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:02.129308 16665 raft_consensus.cc:614] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:21:02.097465 24055 consensus_queue.cc:1243] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:02.087860 23595 consensus_queue.cc:1059] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
W20260709 13:21:02.137372 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 182393us
I20260709 13:21:02.091673 23490 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.143087 15476 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:21:02.146520 15476 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:21:02.147917 24051 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:02.148854 24051 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:02.150211 24051 consensus_queue.cc:1048] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:02.134210 23960 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:02.153117 16744 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:02.153649 16744 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:02.136706 24005 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:02.151333 23972 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:02.155930 23972 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:02.156260 24005 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.156778 23972 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.157963 23972 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.128262 24008 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.167912 23490 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:02.129611 24072 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:02.150756 23788 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.171391 23595 raft_consensus.cc:3055] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:02.171833 23595 raft_consensus.cc:740] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:02.172739 23595 consensus_queue.cc:260] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.173731 23927 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.177956 23960 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:02.179052 23960 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.181178 23595 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:02.173213 23788 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:02.154568 16744 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:02.196031 16744 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:02.188755 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:02.191301 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:02.197312 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:02.192014 16591 raft_consensus.cc:1240] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:02.192910 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:21:02.197417 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44592.8us
I20260709 13:21:02.198429 16737 raft_consensus.cc:2368] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17.
I20260709 13:21:02.201169 17413 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:21:02.192148 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:02.207044 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 283679us
I20260709 13:21:02.208917 17409 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:21:02.218215 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.224623 17413 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:02.225476 16737 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:21:02.229290 16711 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.223208 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:02.236196 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44217us
I20260709 13:21:02.237011 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:02.240010 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:02.241218 17409 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:21:02.240294 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:02.243950 17419 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 13:21:02.246094 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 104774us
I20260709 13:21:02.246955 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:02.235733 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:02.247730 17376 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:02.251415 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.255007 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.256356 17419 raft_consensus.cc:2368] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21.
I20260709 13:21:02.260219 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.261844 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.262661 17415 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:02.263715 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.264477 17410 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.268602 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:02.269304 16744 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 13:21:02.270433 16744 pending_rounds.cc:85] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 11
I20260709 13:21:02.271147 16744 pending_rounds.cc:107] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:21:02.270944 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:02.271499 16744 raft_consensus.cc:2889] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:02.271850 17420 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:02.277467 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:02.288638 17410 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:02.297901 17413 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:02.304479 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:02.308681 16561 leader_election.cc:400] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17.
I20260709 13:21:02.344961 16561 leader_election.cc:403] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:21:02.312263 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.275283 24072 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:21:02.247128 24117 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:02.255808 24109 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:02.275306 23490 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:02.280555 23980 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.318608 24054 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:02.312531 24129 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:21:02.281273 24105 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:21:02.343662 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.322458 17415 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:02.291987 24118 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:02.375257 24141 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:21:02.320039 24073 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:21:02.379175 23927 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:02.379318 24135 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.380085 24135 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.381134 24041 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:02.381803 24041 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:02.349095 24136 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:02.386853 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:02.388031 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.388351 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.388816 17410 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:02.352303 24058 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:02.391324 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:02.403182 17409 raft_consensus.cc:2368] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17.
I20260709 13:21:02.385481 17419 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 12. Current term is 13. Ops: []
I20260709 13:21:02.404855 16589 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
I20260709 13:21:02.413267 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:02.414664 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:21:02.507131 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65745.1us
W20260709 13:21:02.507953 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 70485.8us
I20260709 13:21:02.509868 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:02.511313 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:21:02.515854 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83432.2us
I20260709 13:21:02.508257 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:02.564827 17410 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:21:02.575829 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
W20260709 13:21:02.649855 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 345453us
I20260709 13:21:02.363497 23489 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.140s
I20260709 13:21:02.430276 24054 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:02.656315 16710 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:02.550374 23694 consensus_meta.cc:287] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.526s	user 0.016s	sys 0.002s
I20260709 13:21:02.449446 24073 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:02.653761 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:02.659970 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 90403.5us
I20260709 13:21:02.392030 24118 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:02.664425 16710 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:02.666074 23694 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:02.549775 24117 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:21:02.664155 24129 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.213s	user 0.010s	sys 0.000s
I20260709 13:21:02.649511 23980 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.661397 23869 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.701444 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:02.680091 23946 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.003s
I20260709 13:21:02.721974 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:02.710216 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 481953us
I20260709 13:21:02.568898 24119 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.133s
I20260709 13:21:02.668426 24072 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:21:02.715945 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 204680us
I20260709 13:21:02.747962 16593 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:03.016238 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:02.801970 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 137638us
W20260709 13:21:03.020943 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:02.675601 24005 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:02.693298 24054 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:02.675518 24112 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:03.021456 16742 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:21:02.849858 24144 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:21:02.676990 24073 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:02.823247 23972 consensus_meta.cc:287] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.665s	user 0.013s	sys 0.000s
I20260709 13:21:02.754155 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:02.889210 24140 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.449s	user 0.001s	sys 0.000s
W20260709 13:21:02.849640 24148 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 13:21:02.851148 24041 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:03.035578 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 312542us
I20260709 13:21:02.859516 24135 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:03.033479 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:03.056740 16710 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:02.868765 23811 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.002s
I20260709 13:21:03.039037 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.060655 16668 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:02.813006 24051 raft_consensus.cc:2804] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:03.017764 16561 leader_election.cc:400] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21.
I20260709 13:21:02.822324 24149 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:02.752144 24126 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:21:02.722536 23960 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:02.771096 24117 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.924223 24153 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:02.802132 24109 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:03.049608 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.077404 16668 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:03.077728 17410 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
W20260709 13:21:02.849610 24143 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:21:02.828485 24158 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:03.059767 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:02.923902 24102 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:03.083487 16665 raft_consensus.cc:2393] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 17: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:03.071659 23906 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 1.039s	user 0.013s	sys 0.000s
W20260709 13:21:03.063309 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 45681.4us
I20260709 13:21:02.798099 23918 consensus_queue.cc:1059] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.252s
W20260709 13:21:03.069434 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.019s	sys 0.015s
W20260709 13:21:03.088685 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.020s	sys 0.016s
W20260709 13:21:03.089371 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 581175us
I20260709 13:21:03.087576 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:03.092526 24169 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
W20260709 13:21:03.093502 24139 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:21:03.077613 24117 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.095003 23972 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.095495 23972 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.095877 16743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4548) took 1127 ms. Trace:
I20260709 13:21:03.096529 24117 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.097389 23972 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.096246 16743 rpcz_store.cc:276] 0709 13:21:01.968726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.975893 (+  7167us) service_pool.cc:224] Handling call
0709 13:21:01.976295 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:01.980940 (+  4645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.980972 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:01.981532 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.237099 (+255567us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:02.237210 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.237383 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:03.095686 (+858303us) raft_consensus.cc:1685] finished
0709 13:21:03.095707 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.095823 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4224,"raft.queue_time_us":21393,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":3137,"thread_start_us":246073,"threads_started":1,"wal-append.queue_time_us":390275}
I20260709 13:21:03.078295 24183 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:03.097908 23972 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:02.737010 23869 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.090457 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:03.041805 24073 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:03.099423 24161 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:21:03.101468 24157 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:03.101965 24008 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.102246 24157 consensus_queue.cc:1059] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:03.103505 24167 raft_consensus.cc:3055] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:21:03.103940 24167 raft_consensus.cc:740] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:03.103665 24102 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.098434 24183 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:03.105387 24183 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.106521 24102 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.094566 24135 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:03.080166 24051 raft_consensus.cc:697] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:03.045639 24112 raft_consensus.cc:697] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:21:03.074915 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 320822us
I20260709 13:21:03.055799 24118 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.063966 24041 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.062431 24060 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:03.078779 16559 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:03.062757 17376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.008s	sys 0.011s
I20260709 13:21:03.092463 23918 raft_consensus.cc:3055] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:02.753971 23946 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:03.107577 24058 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.666s	user 0.007s	sys 0.004s
I20260709 13:21:02.821424 24110 consensus_queue.cc:1059] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.235s
I20260709 13:21:03.108357 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:02.813174 23694 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.171036 24110 raft_consensus.cc:3055] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:03.171587 24110 raft_consensus.cc:740] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:03.172742 24110 consensus_queue.cc:260] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.179406 16671 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:03.027918 24005 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:03.180799 24005 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:03.182947 24159 consensus_queue.cc:1175] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 17 status { last_received { term: 17 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 17 index: 13 } }
I20260709 13:21:02.737207 23980 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.185808 23980 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:03.134214 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 46773.6us
I20260709 13:21:03.189409 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.190735 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.191114 24058 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17.
I20260709 13:21:03.192183 16561 leader_election.cc:400] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 16. Current term is 17.
I20260709 13:21:03.192325 24058 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.193063 16561 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:03.193341 24058 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:03.091450 24154 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 13:21:03.098425 23972 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.199708 23869 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.147096 23918 raft_consensus.cc:740] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:03.201421 23972 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.107779 24183 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.201323 23918 consensus_queue.cc:260] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.143215 16669 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:03.203958 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.204624 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.205098 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.205678 16669 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:03.206647 16669 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:03.144104 16666 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:03.144650 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.219875 16668 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:03.146699 17376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.008s	sys 0.011s
W20260709 13:21:03.220613 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 943152us
I20260709 13:21:03.221753 17376 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.149200 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:03.150190 23490 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:03.230080 16744 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.040s	sys 0.056s
I20260709 13:21:03.104806 24167 consensus_queue.cc:260] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.151890 24171 raft_consensus.cc:993] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: : Instructing follower 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:21:03.152215 24051 consensus_queue.cc:237] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.153825 24112 consensus_queue.cc:237] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.131527 24077 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:03.135831 24135 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.155931 16664 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:21:03.155999 16667 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36074 (request call id 3405) took 1121 ms. Trace:
I20260709 13:21:03.079449 24072 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:03.156777 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 78082us
I20260709 13:21:03.157282 24118 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:03.258817 24118 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:03.259395 24167 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:21:03.260331 24006 raft_consensus.cc:1081] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 LEADER]: Signalling peer 3c3ca2a0390b46bb9a5eda03a64eda1a to start an election
I20260709 13:21:03.259953 24118 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.236932 24180 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.148782 24055 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.263238 24055 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.263810 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:03.148133 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.158110 16670 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:03.232923 23788 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.232990 24111 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:03.159525 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.270357 16663 raft_consensus.cc:3060] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:03.145288 24119 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:21:03.162727 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 2150) took 1242 ms. Trace:
I20260709 13:21:03.169600 23946 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.234169 23490 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.143008 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:03.293576 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 150631us
I20260709 13:21:03.266042 24055 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.295547 24187 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:03.295974 24187 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.296334 24187 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.210255 16591 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:21:03.147106 24008 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:03.297540 24008 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.108412 23906 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:03.214303 23595 consensus_meta.cc:287] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.033s	user 0.012s	sys 0.002s
I20260709 13:21:03.149641 24041 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.143491 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:03.310369 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 166959us
I20260709 13:21:03.311354 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:03.223093 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 73987.1us
I20260709 13:21:03.225414 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:03.228655 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:21:03.313181 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 84646.5us
I20260709 13:21:03.081322 24153 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:03.230530 16744 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.040s	sys 0.057s
I20260709 13:21:03.231093 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.174474 24110 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:03.202649 23918 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:03.169762 24131 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.916s	user 0.004s	sys 0.000s
I20260709 13:21:03.236133 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.239203 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
W20260709 13:21:03.320859 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 89843.6us
I20260709 13:21:03.242887 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:03.321888 17376 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.244612 15476 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:21:03.238631 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:03.324049 16736 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
I20260709 13:21:03.324414 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.245942 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:03.246917 16667 rpcz_store.cc:276] 0709 13:21:02.047099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.047623 (+   524us) service_pool.cc:224] Handling call
0709 13:21:03.155723 (+1108100us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b540012b1d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:03.155882 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788146432}
I20260709 13:21:03.253048 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.254732 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.266613 16665 raft_consensus.cc:3055] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:03.326122 16665 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:03.158929 16662 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:03.326951 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.271646 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:03.328796 16596 raft_consensus.cc:2410] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:21:03.272832 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:03.322124 16739 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:03.329293 16595 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:03.329352 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.331979 16588 raft_consensus.cc:2393] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 20: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:03.274505 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:03.333092 17376 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.285147 24151 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.286137 16743 pending_rounds.cc:85] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 10
I20260709 13:21:03.334077 16743 pending_rounds.cc:107] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:21:03.334528 16743 raft_consensus.cc:2889] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:03.286783 24135 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.282436 24162 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.288776 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.289382 24005 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.291306 24111 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:03.291865 24060 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:03.276533 23811 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:03.301448 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.302035 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.302649 16589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d"
dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
 from {username='slave'} at 127.0.0.1:38576
I20260709 13:21:03.307212 16663 raft_consensus.cc:2468] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:03.299930 24008 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.316097 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:03.275701 17411 rpcz_store.cc:276] 0709 13:21:01.919977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.921175 (+  1198us) service_pool.cc:224] Handling call
0709 13:21:01.921663 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:01.922195 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.922239 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:01.974000 (+ 51761us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.197047 (+223047us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:21:02.235725 (+ 38678us) log.cc:844] Serialized 14 byte log entry
0709 13:21:02.247201 (+ 11476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.247289 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:03.158739 (+911450us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b180007e520. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:21:03.162300 (+  3561us) raft_consensus.cc:1685] finished
0709 13:21:03.162444 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.162636 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11223,"prepare.queue_time_us":44980,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":41413,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":63689,"spinlock_wait_cycles":38668544,"thread_start_us":211271,"threads_started":1,"wal-append.queue_time_us":294669}
I20260709 13:21:03.251086 24077 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.365332 16594 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:03.339170 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.356871 24109 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.372869 16667 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:03.344081 24102 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.302374 24105 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.348213 16741 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 21: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:03.349710 16596 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:03.342038 16665 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.380192 16665 raft_consensus.cc:3060] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:03.355247 16589 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:03.355289 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.381814 16589 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:03.382050 16592 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:21:03.359401 16588 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:03.361799 17415 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:03.364142 16669 raft_consensus.cc:1275] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:03.366952 23811 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.368602 24060 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:03.329123 24006 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.360347 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:03.370365 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 54355.2us
I20260709 13:21:03.329313 24180 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:03.333981 24157 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:03.334713 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:03.346005 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:03.399911 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54014.5us
I20260709 13:21:03.400437 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:03.401206 16590 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38576 (request call id 5153) took 1526 ms. Trace:
I20260709 13:21:03.374747 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.375299 16591 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:03.376777 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.404660 17410 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:03.347163 23595 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:03.343433 24185 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.338393 24104 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:21:03.401582 16590 rpcz_store.cc:276] 0709 13:21:01.875261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:01.875412 (+   151us) service_pool.cc:224] Handling call
0709 13:21:01.875721 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:01.876044 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:01.876070 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:01.882541 (+  6471us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b2c000014c8. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327e4dda11 00007f327e4ed372 00007f327fc1fa01 00007f327fbe443a 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2
0709 13:21:01.882961 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:21:02.009934 (+126973us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:02.009975 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:02.010038 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:03.169162 (+1159124us) spinlock_profiling.cc:243] Waited 70.7 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:21:03.400990 (+231828us) raft_consensus.cc:1685] finished
0709 13:21:03.401012 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:03.401148 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":91,"spinlock_wait_cycles":170194688,"thread_start_us":126660,"threads_started":1,"wal-append.queue_time_us":309045}
W20260709 13:21:03.406139 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 71506.4us
I20260709 13:21:03.415778 24072 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:03.379262 24135 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:03.424255 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.369064 24008 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:21:03.389386 24060 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:03.350844 24151 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.356658 24162 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.407481 23489 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:03.423273 23972 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:21:03.398515 24173 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:21:03.388789 24105 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.437568 24135 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.441931 16594 raft_consensus.cc:2393] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 20: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:03.440793 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:03.443984 17376 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.448696 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.407536 23960 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:03.423089 24056 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:03.422420 23980 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:03.458385 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 1945) took 1348 ms. Trace:
I20260709 13:21:03.535856 24157 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.535993 16741 rpcz_store.cc:276] 0709 13:21:02.109872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:02.110010 (+   138us) service_pool.cc:224] Handling call
0709 13:21:03.458009 (+1347999us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400433854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:03.458287 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2837931904}
I20260709 13:21:03.465577 24041 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.463258 24073 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.466475 23906 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.466681 16664 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:21:03.467721 24151 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.434432 24109 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.082s
I20260709 13:21:03.442929 24180 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:21:03.487165 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 46486.9us
I20260709 13:21:03.454761 16589 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:03.517799 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 148365us
I20260709 13:21:03.564757 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.565290 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:03.471220 24135 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:03.475364 24163 consensus_queue.cc:1175] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" responder_term: 14 status { last_received { term: 14 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 14 index: 9 } }
I20260709 13:21:03.471623 24051 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.475710 24150 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.174s
I20260709 13:21:03.552631 23912 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:03.436830 24119 consensus_queue.cc:1059] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 13:21:03.533303 24072 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:03.458946 16738 raft_consensus.cc:1240] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
W20260709 13:21:03.534229 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.469065 24105 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.596055 24151 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:03.606696 24208 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:03.471577 24159 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:03.485644 23489 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.561731 23694 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.526626 24055 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:03.532110 23980 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:03.541256 24186 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.469512 24060 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.479487 24162 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.644375 23489 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:03.627192 24194 log.cc:927] Time spent T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:21:03.627652 24164 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.655285 24164 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.655403 24180 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.657019 23490 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.628298 24104 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:03.659188 24104 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:03.660246 24104 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.661478 24104 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:03.629616 24005 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:03.631057 24190 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:21:03.663069 24117 raft_consensus.cc:3055] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:03.663623 24117 raft_consensus.cc:740] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:03.664498 24117 consensus_queue.cc:260] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.647873 23694 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.665768 24117 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:03.642776 24072 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.667542 23694 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.631155 24187 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.526296 23972 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:03.625126 24051 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.669337 23788 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:03.669776 23788 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:03.670029 23960 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:03.564965 24077 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.670964 23960 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:03.618587 23636 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:21:03.670756 23788 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.672097 23788 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.649174 23980 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:03.671906 23960 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.673382 23960 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.645831 24055 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.662734 24135 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.675704 24055 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:03.676283 24187 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.677183 24135 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.645296 24208 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.599013 23869 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:03.679183 23869 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.679821 24208 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.476147 24178 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.165s
I20260709 13:21:03.645831 24000 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:03.635605 24019 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:03.614059 24041 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:21:03.650986 24073 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.642146 24159 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:03.652067 24153 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.668324 23972 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:03.522681 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 74100.8us
I20260709 13:21:03.781702 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:03.727341 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.782778 16667 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:03.728560 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.541818 24056 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:03.603395 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:03.733354 16589 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.487278 16595 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:21:03.531875 16665 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
W20260709 13:21:03.617067 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49414.8us
I20260709 13:21:03.686084 24019 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:03.532585 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88663.9us
I20260709 13:21:03.812873 24072 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.746229 17419 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:03.757619 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.757280 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.814095 17376 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.686821 24051 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.759708 16671 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:03.820724 16708 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:03.822095 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.681165 23811 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.767347 23869 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.824405 24180 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:03.761852 16664 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:21:03.825132 23489 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.827428 16592 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 25: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:03.782652 23972 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:03.731938 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 363527us
I20260709 13:21:03.730293 16587 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:21:03.803527 24077 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:03.782385 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:03.831054 24077 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.456578 24111 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.693176 24159 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.833593 24072 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.834309 24159 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:03.836665 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 233276us
I20260709 13:21:03.837517 23694 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:03.838555 23694 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.686275 24000 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.843643 24136 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:03.844344 24136 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:03.844569 24178 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.845330 24178 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.684288 24164 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.793129 24208 consensus_queue.cc:1048] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:03.856985 16737 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:21:03.858183 16737 raft_consensus.cc:606] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Received request to transfer leadership
I20260709 13:21:03.820302 24056 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.860595 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:03.861797 17414 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:03.821014 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:03.823303 24212 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:03.863453 16669 raft_consensus.cc:2468] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:21:03.864002 24212 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.776100 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.871337 24178 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.834537 24077 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.853622 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:03.854282 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 72256.2us
I20260709 13:21:03.839195 24150 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:03.854591 23811 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:03.890764 16738 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:03.891127 23811 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:03.891636 23811 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:03.896522 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.897362 17410 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:03.898818 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.899061 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.900444 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.902113 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:03.902779 17409 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:03.905990 24133 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:03.907021 24133 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.907606 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:03.908528 17411 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:21:03.910516 24133 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.898427 24055 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.856297 23972 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.913740 24055 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.914255 23972 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.916695 23972 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.871932 24041 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:03.873045 24060 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.846046 24136 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.919929 24136 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:03.924816 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.878523 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.927505 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.928573 16595 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:03.879489 16591 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a"
I20260709 13:21:03.877452 24185 consensus_queue.cc:1059] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:21:03.929622 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:03.930469 16671 raft_consensus.cc:3055] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:21:03.931017 16671 raft_consensus.cc:740] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:03.932266 16671 consensus_queue.cc:260] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:03.882609 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:03.934741 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:03.937773 16588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.938261 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.940407 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.941207 16594 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:03.941331 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.928587 16665 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:03.877337 16592 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.870657 24212 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.814613 24019 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:03.890179 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:03.890267 16741 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:21:03.943796 16741 raft_consensus.cc:606] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Received request to transfer leadership
I20260709 13:21:03.945245 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
W20260709 13:21:03.945747 16480 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for b40781eb2d304f9b86f5061ae572e1c7 already in progress [suppressed 31 similar messages]
I20260709 13:21:03.926419 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:03.947067 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 126809us
I20260709 13:21:03.949335 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:03.926550 16663 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:21:03.878983 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.930388 24185 raft_consensus.cc:3055] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:03.951231 24133 raft_consensus.cc:2804] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:03.951241 24185 raft_consensus.cc:740] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:03.952162 24133 raft_consensus.cc:697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:03.567970 23906 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:03.953051 24185 consensus_queue.cc:260] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.953480 24133 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:03.954933 23906 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:03.955607 24185 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:03.950649 16595 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:21:03.961624 16671 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:03.958460 23906 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:03.964699 16667 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:03.982306 17414 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:03.900494 24000 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:03.996847 16593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:03.997741 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:04.013101 24072 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:04.016206 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:04.017182 16590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:04.077659 16710 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.079964 24051 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.080466 24051 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:04.080878 24051 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.082938 16740 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:21:04.083407 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.084446 16744 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.085853 23788 raft_consensus.cc:993] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: : Instructing follower ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:21:04.086462 23788 raft_consensus.cc:1081] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Signalling peer ba2df6a668144022a223e504ae571f21 to start an election
I20260709 13:21:04.059854 16665 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:04.043732 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:04.100602 17413 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.046795 16594 raft_consensus.cc:1240] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 26. Current term is 27. Ops: []
I20260709 13:21:04.103317 24077 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.019969 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:04.104008 24077 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:04.104452 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 84588.6us
I20260709 13:21:04.105535 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.106243 24077 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.107259 24117 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:04.107657 24117 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.107802 16595 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:04.108042 24117 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.108913 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.109905 24051 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.110678 16594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:04.111922 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:04.019168 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.029536 17414 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:21:04.030270 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:04.035089 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.038769 16708 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:04.045414 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.045781 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.051185 24055 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
W20260709 13:21:04.065578 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 247210us
I20260709 13:21:04.065696 23980 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:04.123569 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 84940.1us
I20260709 13:21:04.060886 23694 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.027773 16596 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" is_pre_election: true
I20260709 13:21:04.133383 17409 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:04.138782 16667 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:04.143229 16591 raft_consensus.cc:3060] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:04.144523 16738 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:04.118688 16671 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 27.
I20260709 13:21:04.122776 17419 raft_consensus.cc:2393] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:04.129384 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:04.146417 16743 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:04.147078 16708 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:04.139000 16480 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 193397us
I20260709 13:21:04.158100 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.153726 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.133841 24000 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:04.149900 24019 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:21:04.162362 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.172881 16559 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:04.178067 24000 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.212874 16670 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:04.215013 24055 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Leader election won for term 27
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 13:21:04.163578 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:21:04.215721 24055 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:04.181226 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.216415 17416 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:21:04.216655 15476 tablet_server.cc:179] TabletServer@127.15.29.1:0 shutting down...
I20260709 13:21:04.180434 16664 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 14: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:04.185497 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:04.217130 24055 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:04.222265 16561 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:04.161653 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.179442 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.179935 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.191175 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:04.191694 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:04.186124 24118 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:04.221652 24180 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:04.216877 16663 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
W20260709 13:21:04.219539 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 69182.6us
I20260709 13:21:04.193759 24019 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:04.200968 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:04.202943 16596 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:04.233330 16591 raft_consensus.cc:2468] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:04.204988 17420 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:21:04.164883 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.205168 16745 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:21:04.206943 17413 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:04.201993 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.176054 16668 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:04.208030 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.189548 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:04.203526 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.207293 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.199358 17410 raft_consensus.cc:1275] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:21:04.205307 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 70565.9us
I20260709 13:21:04.256181 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:04.202576 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.241025 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:04.074671 23811 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:04.242122 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.261148 16667 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:04.263104 16670 raft_consensus.cc:2368] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18.
I20260709 13:21:04.260257 16740 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:04.231060 24019 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:04.277137 24212 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:04.309648 24212 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.310060 24019 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.311468 24019 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.312065 24212 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.277743 24159 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:04.205710 24172 raft_consensus.cc:2804] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:04.315706 24172 raft_consensus.cc:697] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:04.317119 24172 consensus_queue.cc:237] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:04.192838 23980 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.320149 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.320940 16738 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:21:04.279147 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 88082.4us
I20260709 13:21:04.324268 16710 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:04.279837 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:04.279183 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 186402us
I20260709 13:21:04.266916 24000 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:04.283581 16742 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:04.284242 24008 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:04.203943 17412 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7"
dest_uuid: "ba2df6a668144022a223e504ae571f21"
 from {username='slave'} at 127.0.0.1:48626
I20260709 13:21:04.337468 17412 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:21:04.337823 17412 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.286644 16743 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 13.
I20260709 13:21:04.176707 24051 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.205060 23489 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.197944 23960 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:04.266907 24118 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:04.303887 24058 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:04.308142 16588 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 15: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:04.326510 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:04.327816 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 138446us
W20260709 13:21:04.298408 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 76253.8us
I20260709 13:21:04.335232 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:04.335709 24164 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:04.350637 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 149749us
I20260709 13:21:04.327694 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:04.355373 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.364603 17416 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:21:04.364868 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.365573 16491 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 16 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:04.367173 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.367941 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.368228 24060 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.368637 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.370654 17419 raft_consensus.cc:1240] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:21:04.362437 16741 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:21:04.370508 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.370929 24152 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:04.371284 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.371215 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:04.386185 16668 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:04.371984 24208 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:04.372148 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.367869 16559 leader_election.cc:400] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18.
I20260709 13:21:04.395821 16559 leader_election.cc:403] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:21:04.373093 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.373224 17414 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:21:04.373669 24118 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.374593 24077 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:04.401190 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.374697 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.375293 24058 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:04.379240 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 51658.8us
W20260709 13:21:04.367676 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88003.5us
I20260709 13:21:04.381089 17411 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:04.384177 16744 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:04.398164 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.384794 24060 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:04.382578 23811 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.408195 16745 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:04.411697 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.406774 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:04.403481 24077 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.413820 24077 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:04.411731 24051 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:04.409232 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:04.415638 17410 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 15.
I20260709 13:21:04.415649 24051 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.417356 24073 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.417979 24073 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.420220 24073 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.417380 24153 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:04.421248 24153 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:04.407363 23489 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 17. Current term is 18.
I20260709 13:21:04.422199 24153 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:04.360033 23960 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.423636 24153 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:04.424453 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.424695 23960 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:04.412782 16744 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:21:04.426841 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:04.427208 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.426723 16743 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:21:04.435523 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.436395 16741 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:04.437906 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:04.430540 16743 pending_rounds.cc:85] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Aborting all ops after (but not including) 12
I20260709 13:21:04.451056 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.412675 16491 catalog_manager.cc:5697] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 17 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:04.455118 17411 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:21:04.456518 24073 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:21:04.456766 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52144.2us
I20260709 13:21:04.418700 24051 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.409025 24060 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.441978 16743 pending_rounds.cc:107] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:21:04.445266 17412 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.452570 16739 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:04.461822 16668 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:21:04.469406 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:04.464481 16743 raft_consensus.cc:2889] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:04.399662 24117 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.457783 24072 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:04.461321 24073 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.478834 16670 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:21:04.477900 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 51136.3us
I20260709 13:21:04.488109 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:04.484097 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:04.492888 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.497301 17412 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.501868 16665 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:21:04.473134 24051 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:21:04.465354 23788 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:21:04.477957 23636 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:04.505575 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65105.8us
I20260709 13:21:04.491289 24073 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:04.478787 24008 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:21:04.515113 16590 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: []
I20260709 13:21:04.511936 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.504138 16491 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 17 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:04.500227 23980 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:04.524874 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.526769 16739 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7"
dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:52804
I20260709 13:21:04.527401 16739 raft_consensus.cc:606] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Received request to transfer leadership
I20260709 13:21:04.528126 16738 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 27.
I20260709 13:21:04.519244 24000 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:04.539029 16492 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 234) took 7292 ms. Trace:
I20260709 13:21:04.551093 24167 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:04.556493 24005 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:04.528947 23811 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:04.530752 23869 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:21:04.558419 23972 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:04.529696 23694 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:21:04.562932 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.567982 16744 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:21:04.568033 16745 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:04.571223 24005 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.573302 17411 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
W20260709 13:21:04.591332 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.592904 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.584743 24178 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.597283 24178 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.599139 24005 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.600818 24178 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:04.603852 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 115875us
I20260709 13:21:04.498821 24058 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.587028 23694 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:04.585043 23811 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:04.616645 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:04.614476 23694 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:04.626783 16561 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:04.638360 24208 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:04.564229 16492 rpcz_store.cc:276] 0709 13:20:57.297867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:57.312961 (+ 15094us) service_pool.cc:224] Handling call
0709 13:20:57.429933 (+116972us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:21:04.538742 (+7108809us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:02.383055 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:02.383767 (+   712us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:02.383808 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:21:02.383829 (+    21us) tablet.cc:662] Decoding operations
0709 13:21:02.786030 (+402201us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:02.795244 (+  9214us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:02.797703 (+  2459us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:21:02.915376 (+117673us) tablet.cc:701] Row locks acquired
0709 13:21:02.926140 (+ 10764us) write_op.cc:260] PREPARE: finished
0709 13:21:02.926310 (+   170us) write_op.cc:270] Start()
0709 13:21:02.926467 (+   157us) write_op.cc:276] Timestamp: P: 1783603262926262 usec, L: 0
0709 13:21:02.926489 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:21:03.071443 (+144954us) log.cc:844] Serialized 5927 byte log entry
0709 13:21:03.200543 (+129100us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbdfe28 00007f328361a581 00007f3283619d58 00007f328361e9f1 00007f328361e9a9 00007f328361e939 00007f328361e901 00007f328361db2d 000000000038f354 000000000038f199
0709 13:21:03.463215 (+262672us) op_driver.cc:464] REPLICATION: finished
0709 13:21:03.536622 (+ 73407us) write_op.cc:301] APPLY: starting
0709 13:21:03.536747 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:03.609087 (+ 72340us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:03.609125 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:03.674111 (+ 64986us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:03.674430 (+   319us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 13:21:03.674481 (+    51us) write_op.cc:312] APPLY: finished
0709 13:21:03.674847 (+   366us) log.cc:844] Serialized 139 byte log entry
0709 13:21:03.685211 (+ 10364us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:03.691828 (+  6617us) write_op.cc:454] Released schema lock
0709 13:21:03.693800 (+  1972us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:03.696121 (+  2321us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:03.696157 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":106240,"reported_tablets":28,"spinlock_wait_cycles":30192512,"child_traces":[["op",{"apply.queue_time_us":73102,"apply.run_cpu_time_us":25588,"apply.run_wall_time_us":158236,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66887,"num_ops":15,"prepare.queue_time_us":229199,"prepare.run_cpu_time_us":27116,"prepare.run_wall_time_us":828297,"raft.queue_time_us":149630,"raft.run_cpu_time_us":11840,"raft.run_wall_time_us":164972,"replication_time_us":536523,"spinlock_wait_cycles":126820736,"thread_start_us":325452,"threads_started":4,"wal-append.queue_time_us":167090}]]}
I20260709 13:21:04.639043 16739 raft_consensus.cc:2468] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 10.
I20260709 13:21:04.652213 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.653759 17411 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:04.656590 23788 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:04.657850 23788 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.658607 23788 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.663851 17409 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:04.659395 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:04.664453 17411 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:21:04.664801 17412 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:04.651513 17374 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:04.620904 24005 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 13:21:04.673781 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47118.2us
I20260709 13:21:04.682488 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.680763 24117 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.685277 24178 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:04.709977 24060 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.720867 16491 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:04.712589 23788 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.724472 16491 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 17 to 20, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 20 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:04.722784 23489 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.726122 23489 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:04.702242 24178 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.727147 24178 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:04.735811 16742 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 25.
I20260709 13:21:04.730054 16491 catalog_manager.cc:5697] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 20 to 22, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 22 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:04.718591 24111 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:21:04.750077 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:04.716558 24167 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:04.684345 23811 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:04.803421 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:04.813216 16662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.024s	sys 0.044s
W20260709 13:21:04.813715 16662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.025s	sys 0.045s
I20260709 13:21:04.819343 23960 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:04.820164 23960 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:04.821233 24133 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.037s	sys 0.069s
W20260709 13:21:04.821655 24133 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.037s	sys 0.069s
I20260709 13:21:04.823915 23489 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.824829 16737 raft_consensus.cc:1240] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:04.799556 24111 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:04.848114 17376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.033s	sys 0.033s
W20260709 13:21:04.848790 17376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.033s	sys 0.033s
I20260709 13:21:04.802942 24000 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.863229 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:04.864289 17409 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:04.833796 23960 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.834497 24178 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:04.889703 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 425659us
I20260709 13:21:04.879122 23960 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:04.895645 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 299108us
W20260709 13:21:04.897425 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 373344us
I20260709 13:21:04.892549 16710 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:04.893227 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 375609us
I20260709 13:21:04.875811 24000 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.906703 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:04.907084 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:04.907609 16670 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:04.907761 16668 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:04.920066 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 284139us
I20260709 13:21:04.906917 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:04.894106 23980 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.922734 16559 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:04.903949 23906 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:04.890734 24077 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.901036 24072 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:04.905469 24060 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.919178 24005 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:04.904951 23489 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 13:21:04.897608 23960 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.940578 17409 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:21:04.943313 16737 raft_consensus.cc:1240] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 11. Current term is 12. Ops: []
W20260709 13:21:04.943667 24019 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.001s
I20260709 13:21:04.946497 16668 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:04.950315 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.941867 24153 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:04.947855 23636 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.968954 24231 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:04.969456 24231 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:04.969960 24231 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:04.973748 24000 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:04.973836 16738 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:04.974232 24077 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:04.966778 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:04.966146 24058 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:04.982739 24058 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:04.983132 24104 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:04.983251 24058 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:04.983744 24104 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:04.984280 24104 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:04.986028 24224 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.982749 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:04.987187 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:04.987474 24185 raft_consensus.cc:2764] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:21:04.986940 24224 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:04.988440 24223 log.cc:927] Time spent T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:21:04.989293 24133 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.990701 24229 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.991348 24229 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:04.928511 24111 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:04.995604 24225 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:21:04.980232 23636 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.953693 23906 raft_consensus.cc:697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:04.964812 24060 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:04.974907 24053 raft_consensus.cc:3055] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:04.955003 23980 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:04.955225 24152 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
W20260709 13:21:05.018679 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:05.032435 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49777.1us
W20260709 13:21:05.046425 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.045737 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:05.043748 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 109472us
W20260709 13:21:05.047442 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:04.964334 24005 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.030689 24053 raft_consensus.cc:740] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:21:05.041852 24073 consensus_meta.cc:287] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.004s
I20260709 13:21:05.052274 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:05.048132 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:04.989912 24133 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.054693 17413 raft_consensus.cc:1240] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:05.056171 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.060232 17420 raft_consensus.cc:2368] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20.
I20260709 13:21:05.062326 17414 raft_consensus.cc:1275] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:21:05.080506 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.062985 17410 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
W20260709 13:21:05.063504 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:05.081310 16741 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:21:05.044028 24060 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.057622 24164 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:05.056392 24153 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:05.099540 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.091487 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.100461 17420 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:05.100705 24224 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.092803 24208 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:21:05.058836 24185 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:05.093884 24005 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.098795 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.099619 24153 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:05.102222 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.096909 24229 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:05.100872 24053 consensus_queue.cc:260] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.104879 24185 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.104980 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:05.140381 16737 raft_consensus.cc:3055] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:05.101794 24178 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.140856 16737 raft_consensus.cc:740] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:05.091176 24152 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:05.105873 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:05.121953 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:05.128070 24226 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:21:05.126844 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.163249 16742 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 12.
I20260709 13:21:05.062501 24073 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.152760 16736 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
W20260709 13:21:05.153009 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:05.168864 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:05.169656 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:05.170282 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:05.151342 24185 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:05.170807 16634 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:05.172034 24152 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:05.117029 24208 consensus_queue.cc:1243] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer 3c3ca2a0390b46bb9a5eda03a64eda1a log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:21:05.168869 16737 consensus_queue.cc:260] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.184396 24152 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.086462 23906 consensus_queue.cc:237] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.088462 24072 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:05.212877 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 149197us
I20260709 13:21:05.218272 16708 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:05.229212 24239 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:21:05.230628 24153 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.231276 24224 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:05.232164 24153 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.238373 24243 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.238566 23972 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.239159 23972 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:05.185034 24208 consensus_queue.cc:1059] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:21:05.211556 23636 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.234378 24231 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.247249 24060 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.260258 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:05.250296 23960 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:05.252213 24133 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.253997 24248 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.270107 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:05.255622 24073 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:05.261034 23694 consensus_meta.cc:287] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.614s	user 0.016s	sys 0.015s
W20260709 13:21:05.272418 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 163391us
I20260709 13:21:05.278218 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:05.279245 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 142724us
I20260709 13:21:05.261493 23980 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:05.355849 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 70510us
I20260709 13:21:05.273403 24133 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:05.259812 24243 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.276320 23694 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.284792 24104 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:05.391445 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 164494us
I20260709 13:21:05.394059 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.275954 24248 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.394902 17414 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
W20260709 13:21:05.400166 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 122385us
I20260709 13:21:05.400512 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:05.402453 16710 leader_election.cc:400] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20.
I20260709 13:21:05.403236 16710 leader_election.cc:403] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:21:05.396533 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:05.406692 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 132550us
I20260709 13:21:05.407164 16710 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:05.397542 24133 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:05.414901 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.391120 24208 consensus_queue.cc:1048] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:21:05.415613 17416 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 12.
I20260709 13:21:05.418852 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:05.447854 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.417207 17376 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:05.475739 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:05.403421 23906 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:21:05.420928 24235 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:05.479512 24238 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:21:05.510557 24055 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.481447 24152 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.514953 24251 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:05.515439 24251 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.515081 24152 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.515818 24251 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.516145 24073 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.516352 24261 consensus_queue.cc:1048] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:05.516738 24073 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.517819 23694 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:05.519176 24245 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:21:05.520179 24266 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.520705 24247 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:21:05.520973 24266 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.514498 24253 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:05.498633 24231 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.499084 23811 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.511937 24241 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.513656 24235 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.526342 24253 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:05.530117 23906 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.536739 23960 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:05.550714 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 129966us
I20260709 13:21:05.532294 24073 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.552162 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:05.550985 24058 consensus_meta.cc:287] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.567s	user 0.022s	sys 0.005s
I20260709 13:21:05.545768 24235 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:05.548709 24253 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:05.542198 24055 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:05.567111 24252 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:21:05.568400 16542 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:21:05.576064 16542 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for b40781eb2d304f9b86f5061ae572e1c7 already in progress
I20260709 13:21:05.557153 23906 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.576675 24243 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:05.552158 23811 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.577100 24250 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:05.578500 24257 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:21:05.560086 23960 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.566608 24058 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.583853 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:05.604884 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:05.576370 16542 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for b40781eb2d304f9b86f5061ae572e1c7 already in progress
W20260709 13:21:05.614223 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.615049 24274 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.616631 24274 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.590734 24267 consensus_queue.cc:1059] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:21:05.623184 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:05.577143 24256 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:21:05.579517 24260 log.cc:927] Time spent T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:21:05.623862 17416 raft_consensus.cc:2468] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 12.
I20260709 13:21:05.578584 23906 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:05.580207 24055 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.578815 23811 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:05.623549 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:05.583763 24077 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:05.630338 24243 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.630782 17416 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:21:05.631247 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.630818 24243 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.631865 17411 raft_consensus.cc:2468] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:05.633185 24243 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.551982 24241 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.635147 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:05.636373 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 52606.5us
I20260709 13:21:05.622162 24242 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:05.639464 24242 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:05.639918 24242 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:05.640875 24254 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.641404 24254 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.641754 23489 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.641901 24248 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.650535 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:05.651475 17411 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:05.651880 24266 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.627789 24104 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.645884 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.656132 17414 raft_consensus.cc:2468] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:05.658107 16738 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:05.624954 23972 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.658629 17416 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:05.642546 24248 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:05.667065 17411 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:05.660456 23980 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.650233 24060 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:05.643803 23811 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.642335 23489 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.627061 24185 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:05.688426 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 49140.5us
I20260709 13:21:05.645485 23694 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.643850 24077 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.714838 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:05.715553 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.716428 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.660598 23906 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.712056 23980 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.721493 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.691751 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.720746 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.717217 24060 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.691862 24133 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:05.726761 24259 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.177s	user 0.001s	sys 0.001s
I20260709 13:21:05.727159 24133 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.727478 24060 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.728152 16743 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:21:05.722124 16736 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:05.712970 24005 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.753958 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:05.755405 16736 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:05.780133 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:05.780889 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:05.732795 24073 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.732497 24152 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.733054 24055 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:05.784757 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 60850.6us
I20260709 13:21:05.784943 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:05.712810 23811 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.743287 24056 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:05.717605 23694 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.788555 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.730291 24077 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.790750 16745 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:05.751024 24060 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.731257 23906 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:05.793502 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.797151 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:05.799423 16668 raft_consensus.cc:2393] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:05.800379 16740 raft_consensus.cc:1240] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:05.815534 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:05.834903 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 46914.2us
I20260709 13:21:05.815637 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:05.836445 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:05.778784 24005 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.800873 23636 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:05.848934 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:05.788754 24073 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.790932 24055 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.814503 24058 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.795526 24274 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.871608 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:05.861361 24255 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:21:05.850390 23960 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.874536 23972 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:05.912462 16736 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:05.877204 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:05.813949 24060 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.927402 24060 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:05.838563 24056 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:05.930250 24056 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:05.931380 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 54268.4us
I20260709 13:21:05.932287 16561 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:05.933444 23869 raft_consensus.cc:2804] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:21:05.933992 23869 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:05.934497 23869 raft_consensus.cc:3060] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:21:05.943292 24056 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:05.944207 24056 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:05.945516 24056 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:05.991707 24229 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:21:05.992380 24229 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:05.993698 24229 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.009282 24229 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:06.017228 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:05.855619 24281 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.862021 24254 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:06.079586 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:05.879899 24058 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:05.857565 23636 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:06.114526 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 278141us
W20260709 13:21:06.115067 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 316725us
I20260709 13:21:06.046144 24243 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.079111 24185 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.110607 24269 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:06.044685 24060 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.078s
W20260709 13:21:06.139626 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 99694us
I20260709 13:21:06.120309 23980 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.125563 23489 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.126350 24241 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:06.124835 24258 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:06.138521 24276 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.346s	user 0.001s	sys 0.001s
I20260709 13:21:06.136423 24269 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.081163 24281 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:06.176070 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 55055.5us
I20260709 13:21:06.127975 24058 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.134454 24243 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:06.163836 24247 consensus_meta.cc:287] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.005s
I20260709 13:21:06.162493 24077 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.152205 24133 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:06.171314 23811 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.192557 23694 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:06.212612 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 162406us
I20260709 13:21:06.213429 24282 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:06.151513 24000 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.170620 24231 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:06.172968 24263 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.171778 24224 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:21:06.212102 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Timed out: RequestConsensusVote RPC to 127.15.29.2:42639 timed out after 1.605s (SENT)
I20260709 13:21:06.164666 24056 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:21:06.246987 16710 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:06.172705 24269 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:06.212903 24247 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20.
I20260709 13:21:06.225704 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:06.303995 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 78458.5us
I20260709 13:21:06.305073 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:06.211951 24287 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.308722 24287 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.309587 24281 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.228320 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.213526 24236 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.229517 16662 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:06.230252 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:06.215494 23489 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.315253 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:06.232636 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.233201 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.174293 24258 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.212431 23980 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:06.237149 23694 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:21:06.237331 24231 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.238709 24263 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.251037 24159 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:06.229122 24282 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.333662 24159 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.333820 24282 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:06.334287 24159 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:06.268793 24243 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.341766 24231 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.261511 24055 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.272167 23636 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.272531 24224 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:06.273830 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 61753.8us
I20260709 13:21:06.274518 24273 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.274118 23869 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.234440 24133 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.225360 24005 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.248196 24000 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.225757 24152 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.246604 24268 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.311426 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:06.315537 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 85354.7us
I20260709 13:21:06.308149 24056 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:06.329200 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.331384 24258 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:06.331991 16662 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:06.312608 24289 raft_consensus.cc:2804] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:21:06.332656 24263 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.332479 23489 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.331399 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.431035 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:06.435348 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 123992us
I20260709 13:21:06.441298 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:06.342742 24235 consensus_meta.cc:287] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.000s
W20260709 13:21:06.347407 24250 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.770s	user 0.011s	sys 0.000s
I20260709 13:21:06.447424 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:21:06.351039 24253 consensus_meta.cc:287] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.775s	user 0.013s	sys 0.000s
I20260709 13:21:06.445744 16670 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: []
W20260709 13:21:06.344276 24251 consensus_meta.cc:287] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.828s	user 0.013s	sys 0.001s
I20260709 13:21:06.446568 16744 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:06.443254 16669 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:06.315946 24281 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 13:21:06.448737 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:06.450254 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.450634 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.452049 23489 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.452486 24111 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:06.459102 24060 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:06.463209 24224 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:06.447892 24235 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.465510 24291 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.448522 23694 raft_consensus.cc:2804] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:06.467607 24000 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.439339 24056 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:06.439361 24152 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.472417 16737 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:06.472573 24053 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:06.477624 24172 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:06.479203 23811 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.475288 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.481148 24058 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.542039 24153 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:21:06.490265 24242 consensus_meta.cc:287] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.850s	user 0.008s	sys 0.003s
I20260709 13:21:06.497613 23972 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:06.463557 23906 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:06.501732 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 894a0fdf94344f46b0426efbd37a6c40 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ba2df6a668144022a223e504ae571f21 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:06.503791 17420 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:21:06.548444 23489 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:06.547667 24060 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:06.537416 16669 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:06.539024 16667 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:06.540653 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:06.580641 17412 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:06.476927 24159 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.541844 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:06.531244 24263 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:06.318686 24236 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.569993 24172 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.592036 24236 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:06.326643 23980 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.446605 24055 raft_consensus.cc:2804] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:06.544617 16742 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:21:06.567802 24152 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.434504 24268 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.557858 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:06.353061 24287 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.546027 24270 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:21:06.608053 17374 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.731s (SENT)
I20260709 13:21:06.545606 23694 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.560431 24279 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:06.518436 24250 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:06.434617 24133 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.434273 24289 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:06.569407 16670 raft_consensus.cc:1275] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:06.545414 24235 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.574780 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:06.575811 16480 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 74143.3us
I20260709 13:21:06.504968 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.570204 23811 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.561123 24243 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.581768 17416 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:06.434561 24273 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.630745 24270 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.543591 24253 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.631546 16663 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 13:21:06.578280 24242 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.633493 24253 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.634339 24242 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.583137 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:06.583907 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:06.532820 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 84229us
I20260709 13:21:06.580518 23972 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:06.551256 16634 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:06.545542 24291 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.604967 16710 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:06.539472 24251 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.627779 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:06.600065 24055 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.610888 23694 raft_consensus.cc:3060] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:06.624161 24289 raft_consensus.cc:3060] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:06.634750 24243 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.635957 24295 consensus_queue.cc:1048] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:21:06.582729 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:06.676793 16669 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:06.639369 16744 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 1961) took 1220 ms (client timeout 1626 ms). Trace:
I20260709 13:21:06.630571 24279 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.639887 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:06.648849 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 97606.1us
I20260709 13:21:06.575700 24282 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:06.654978 16662 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:06.656881 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 51993us
I20260709 13:21:06.654469 24172 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.673627 16670 raft_consensus.cc:2368] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 22. Current term is 23.
I20260709 13:21:06.673260 16737 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:06.671929 24287 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:06.656461 24253 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:06.679371 24270 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.716432 24273 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.673368 24055 raft_consensus.cc:3060] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:06.679883 23869 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.680794 24159 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.685120 16665 raft_consensus.cc:3060] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:06.677564 16744 rpcz_store.cc:269] 0709 13:21:05.418442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.418608 (+   166us) service_pool.cc:224] Handling call
0709 13:21:06.639170 (+1220562us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b54003eeed4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:06.639283 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061107456}
I20260709 13:21:06.600335 24056 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.689507 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:06.690208 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:06.691766 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:06.692409 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:06.744022 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51658.5us
W20260709 13:21:06.709638 16737 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41680 (request call id 1957) took 1611 ms (client timeout 1731 ms). Trace:
I20260709 13:21:06.672799 24235 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:06.734813 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:06.746515 16737 rpcz_store.cc:269] 0709 13:21:05.098044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.104682 (+  6638us) service_pool.cc:224] Handling call
0709 13:21:06.471310 (+1366628us) spinlock_profiling.cc:243] Waited 90.5 ms on lock 0x7b58004be6d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe827c 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c
0709 13:21:06.709542 (+238232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208170880}
I20260709 13:21:06.754390 17415 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 13: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:06.653728 24269 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.755694 16669 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:06.757511 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.727888 24267 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.764314 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.773329 16671 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:06.767791 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.767688 16670 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
W20260709 13:21:06.768996 16634 leader_election.cc:336] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:06.774516 17410 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48618 (request call id 1639) took 1396 ms. Trace:
I20260709 13:21:06.782680 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.783409 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.783427 16662 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:06.783679 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.785879 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.782577 17410 rpcz_store.cc:276] 0709 13:21:05.375357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.375561 (+   204us) service_pool.cc:224] Handling call
0709 13:21:05.378476 (+  2915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:05.688276 (+309800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:05.688309 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:05.688361 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:05.688407 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.771179 (+1082772us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400aa55d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:06.771226 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:06.771360 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642475776}
W20260709 13:21:06.787024 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59898.3us
I20260709 13:21:06.749089 24295 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.788980 16666 raft_consensus.cc:1240] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:21:06.749627 24235 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:06.730728 24287 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.736995 24253 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:06.853295 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 53011.9us
I20260709 13:21:06.685772 24060 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.771639 24051 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.856086 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.791003 24236 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.797423 24000 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.815578 24235 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:06.881374 17416 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:21:06.884066 16662 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 17. Current term is 18. Ops: []
W20260709 13:21:06.858315 24229 consensus_meta.cc:287] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.844s	user 0.013s	sys 0.003s
I20260709 13:21:06.890486 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.853719 23869 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.856769 23906 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.782799 24267 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.916953 16665 raft_consensus.cc:2468] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:06.934595 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:06.932843 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:06.842247 24295 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.966411 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:06.971303 16743 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:21:06.967104 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:06.971760 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:06.972141 16744 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:06.973711 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:06.974293 16665 raft_consensus.cc:2376] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19. Re-sending same reply.
I20260709 13:21:06.887378 24281 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:06.978534 16745 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:06.974108 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:06.981822 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49051us
I20260709 13:21:06.891312 23788 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:06.983814 16559 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:06.942660 24224 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.924861 24172 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:06.925202 24282 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.943338 24242 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.943670 24005 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:07.003518 24264 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:06.979005 23906 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.919338 24000 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.007071 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.009331 24073 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.019603 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.088294 16664 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:07.020495 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.020843 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:07.020537 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:06.897116 24051 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.991995 24058 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:06.881740 24298 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:06.929394 23869 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:06.996286 24279 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:06.927428 24236 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.099690 23869 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:06.993413 23972 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.100731 24236 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.096702 24298 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.104990 24298 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.093796 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.106936 16740 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
W20260709 13:21:07.090822 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 156330us
I20260709 13:21:07.112021 17376 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:06.996003 23788 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.106244 23489 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:07.031116 24172 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.128067 16745 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:07.080818 24242 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.028517 24281 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.121937 23980 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.084342 24005 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.029645 24224 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.084553 24264 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.128021 24058 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:07.148021 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:07.257963 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:07.259248 17416 raft_consensus.cc:1275] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:07.346379 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.346668 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.263878 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:07.347316 16744 raft_consensus.cc:2468] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:21:07.264173 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:07.348133 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.348429 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.348575 17420 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.348781 16741 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:07.349169 16744 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:07.267112 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:07.350310 17413 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:07.264540 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:07.353016 17411 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:07.347110 16745 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.120314 24291 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.269735 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:07.289386 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.289740 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:07.301064 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 189120us
I20260709 13:21:07.364461 16665 raft_consensus.cc:2393] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 24: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:07.311117 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:07.306118 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3448) took 1058 ms. Trace:
I20260709 13:21:07.114856 24282 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:21:07.276154 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41680 (request call id 1963) took 1857 ms. Trace:
I20260709 13:21:07.347594 17419 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:07.305634 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.300235 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.365226 16666 rpcz_store.cc:276] 0709 13:21:06.247577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.247746 (+   169us) service_pool.cc:224] Handling call
0709 13:21:06.250957 (+  3211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:06.843143 (+592186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.305863 (+462720us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b540023d054. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:07.306051 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236652288}
I20260709 13:21:07.289108 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.297266 24309 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:21:07.173503 24301 log.cc:927] Time spent T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.306s	user 0.002s	sys 0.000s
I20260709 13:21:07.262354 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:07.445819 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.543828 16662 raft_consensus.cc:2410] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:21:07.544775 16736 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:07.446380 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.446941 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.537412 16742 rpcz_store.cc:276] 0709 13:21:05.419096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.419208 (+   112us) service_pool.cc:224] Handling call
0709 13:21:05.449116 (+ 29908us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:06.596688 (+1147572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.275778 (+679090us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003d0254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:07.276041 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2518963456}
I20260709 13:21:07.451447 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:07.216385 24053 consensus_meta.cc:287] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.659s	user 0.010s	sys 0.001s
W20260709 13:21:07.550097 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 287850us
I20260709 13:21:07.455829 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.166075 24073 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.556588 16667 raft_consensus.cc:2410] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:21:07.194187 24051 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.203460 24224 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.301342 24314 raft_consensus.cc:2804] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:07.160929 24269 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:07.359548 24289 consensus_meta.cc:287] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.656s	user 0.008s	sys 0.005s
I20260709 13:21:07.191226 24055 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.467767 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.195308 24000 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.223984 24295 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.583178 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.219228 24267 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.241227 24133 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.200208 23788 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.584755 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.196199 23906 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.154482 24281 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.589191 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.164139 24268 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.223559 24005 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.180231 24264 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.255548 24251 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.235219 24287 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:07.480027 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 484710us
I20260709 13:21:07.601861 16559 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:07.299743 23811 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.490527 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.603430 16665 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:21:07.495257 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.295469 24235 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.508917 24315 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:07.510985 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 211820us
I20260709 13:21:07.512363 24320 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:07.518582 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 154073us
W20260709 13:21:07.523527 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 235731us
I20260709 13:21:07.591063 23636 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.583191 24314 raft_consensus.cc:697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:07.365280 17415 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:07.294215 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3447) took 1057 ms. Trace:
I20260709 13:21:07.547622 16669 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:07.545845 16739 raft_consensus.cc:2468] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:21:07.555473 24306 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:07.461596 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.201435 24242 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.344091 24164 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:07.570878 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.293685 24153 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:07.300330 24308 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:07.204609 24172 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.188525 24253 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.299611 24072 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:07.577086 16668 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:07.249711 24279 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:07.628093 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.544114 24309 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.293123 24243 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:07.581621 17374 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 2.145s (SENT)
I20260709 13:21:07.583868 16742 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.584302 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.548873 24291 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.558617 24152 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:07.257889 23694 consensus_meta.cc:287] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.580s	user 0.009s	sys 0.001s
I20260709 13:21:07.639266 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.608385 24267 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:07.639753 23694 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.640748 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.640646 24267 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.642144 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.576457 24224 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.591722 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.603811 16737 raft_consensus.cc:2468] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:21:07.634572 16710 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.643224 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.607605 24005 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:07.536029 16745 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:07.576014 24073 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:07.645979 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 894a0fdf94344f46b0426efbd37a6c40 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ba2df6a668144022a223e504ae571f21 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:07.644024 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:07.647332 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55690us
I20260709 13:21:07.647722 16737 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:07.644017 16665 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:07.649284 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.617419 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.650599 16670 raft_consensus.cc:2410] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:21:07.622606 24315 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.651758 24056 raft_consensus.cc:2804] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:07.652271 24056 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.652700 24056 raft_consensus.cc:3060] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.653406 24315 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.618466 24320 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.645272 24306 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:07.628974 24164 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.655958 24320 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.655879 24306 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.620781 16744 raft_consensus.cc:2468] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 15.
I20260709 13:21:07.657049 24164 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.657469 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.625505 24153 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:07.658654 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:07.659595 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:07.660207 16634 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:07.661339 16634 leader_election.cc:336] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.661769 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.621043 16663 rpcz_store.cc:276] 0709 13:21:06.236853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.236991 (+   138us) service_pool.cc:224] Handling call
0709 13:21:06.271332 (+ 34341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:06.631990 (+360658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:07.293951 (+661961us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54001b08d4. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:07.294127 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827535104}
I20260709 13:21:07.618492 24269 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:07.663237 24269 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:07.664543 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50893.2us
I20260709 13:21:07.593158 24133 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:07.579955 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.665166 24133 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.665336 16740 raft_consensus.cc:2468] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:07.665683 24133 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:07.666342 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.666976 16740 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:07.667742 24055 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.668408 24055 consensus_queue.cc:1059] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:07.669273 24298 raft_consensus.cc:3055] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:07.669643 24298 raft_consensus.cc:740] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:07.669831 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.670558 16745 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:07.670452 24298 consensus_queue.cc:260] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.632050 24309 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.671520 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.671645 24298 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:07.635293 24072 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.672689 16739 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:07.673760 24072 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.674367 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.629454 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:07.675238 16666 raft_consensus.cc:2376] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20. Re-sending same reply.
I20260709 13:21:07.676191 24235 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.675951 24314 consensus_queue.cc:237] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.622455 24281 consensus_queue.cc:1059] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:21:07.678047 24281 raft_consensus.cc:3055] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:07.678419 24281 raft_consensus.cc:740] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:07.636341 24308 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.645761 24224 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.677608 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.679195 24281 consensus_queue.cc:260] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.680459 24281 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.649684 24208 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:07.681300 16739 raft_consensus.cc:2393] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:07.681561 24208 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.681759 17416 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:07.682025 24208 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:07.682119 24269 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.682539 16736 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.679167 24250 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.683245 24250 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.628018 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.684719 16664 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:21:07.631896 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.685249 24250 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:07.632097 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 50536.8us
I20260709 13:21:07.588471 24051 consensus_queue.cc:1048] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:21:07.636485 16743 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:07.636251 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:07.630992 16708 leader_election.cc:336] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.637629 24279 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.626801 23636 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.699463 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.638713 24291 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.642259 23694 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.611191 24273 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.649312 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:07.662112 16634 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.681011 24224 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.577723 24289 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.716651 24289 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.630214 17409 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 2198) took 1128 ms. Trace:
I20260709 13:21:07.717630 24279 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.717269 17409 rpcz_store.cc:276] 0709 13:21:06.501868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:06.505810 (+  3942us) service_pool.cc:224] Handling call
0709 13:21:06.541302 (+ 35492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:06.551285 (+  9983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:06.551317 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:06.559707 (+  8390us) log.cc:844] Serialized 26 byte log entry
0709 13:21:06.603924 (+ 44217us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:06.603966 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:06.604039 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:07.629944 (+1025905us) raft_consensus.cc:1685] finished
0709 13:21:07.629978 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:07.630137 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":82,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":9766,"threads_started":1,"wal-append.queue_time_us":164741}
I20260709 13:21:07.687372 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.637533 24152 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.667774 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.709909 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.721457 16741 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:07.591442 17420 raft_consensus.cc:2468] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:07.690455 16742 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:07.613101 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:07.687383 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56420.7us
I20260709 13:21:07.177285 24058 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.682232 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.696544 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.681011 24308 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:07.726824 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 90654us
I20260709 13:21:07.726563 16671 raft_consensus.cc:2393] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 16: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:07.727881 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:21:07.697413 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:07.723322 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 61228.8us
I20260709 13:21:07.729096 17411 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:21:07.697841 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:07.727324 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.698472 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:07.724061 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.700878 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.699841 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:07.709935 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60705.7us
I20260709 13:21:07.731009 16664 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:07.733654 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:07.711058 23811 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:07.737978 16736 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:07.711366 23980 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:07.712838 24267 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.688913 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:07.735002 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:07.724673 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:07.697189 24051 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:07.727506 17376 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.701642 16744 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 25.
I20260709 13:21:07.726537 24250 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.747186 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:07.752198 16742 raft_consensus.cc:1240] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:07.751694 16710 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.754590 16669 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:21:07.758124 16741 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:21:07.737463 24073 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.766570 23636 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.726550 24321 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:07.767688 24324 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:07.767782 24321 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.768785 24324 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:07.749589 24060 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:07.769235 24324 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:07.769361 24060 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.769841 24060 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:07.774379 24243 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:07.745957 23811 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.774950 24243 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.775131 24055 raft_consensus.cc:2804] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:07.776000 24055 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.776592 24055 raft_consensus.cc:3060] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:07.778995 24072 raft_consensus.cc:2804] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:07.736799 24242 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:07.779928 24072 raft_consensus.cc:697] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:07.780047 24242 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:07.758085 24250 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.781443 24242 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.781526 24072 consensus_queue.cc:237] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.784309 16743 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:07.775456 24243 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:07.778506 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:07.789343 16634 leader_election.cc:336] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.756619 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:07.805472 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 48953.7us
I20260709 13:21:07.806348 16561 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:07.791239 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.728367 24273 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.775182 23811 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:07.807709 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.810338 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:07.803480 16634 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.816958 24164 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:07.831086 24164 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:07.831691 24164 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:07.819870 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.821895 24159 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:07.836591 16561 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.837472 24321 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.838158 24310 raft_consensus.cc:2804] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:07.838712 24310 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.839164 24310 raft_consensus.cc:3060] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:07.839785 23489 raft_consensus.cc:2804] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:07.840267 23489 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.840693 23489 raft_consensus.cc:3060] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:07.840782 24273 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:07.842013 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:07.843078 24289 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:07.842975 24153 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.843776 24289 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:07.845031 24264 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:07.812444 23694 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:07.844821 24289 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:07.845566 24153 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:07.844645 24208 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.770280 24324 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:07.854588 16663 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:21:07.847951 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:07.851478 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.856467 16669 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:07.812646 24005 raft_consensus.cc:2804] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:07.869935 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.873157 24159 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:07.873755 24159 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:07.874493 24159 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:07.880915 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.882910 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.883761 16736 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:07.886469 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.888041 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:07.894950 16745 raft_consensus.cc:2393] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 20: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:21:07.884243 16634 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.896020 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:07.896816 16669 raft_consensus.cc:2393] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 19: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:07.903985 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.907588 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.912200 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:07.926241 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:07.938593 16634 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.938665 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.941166 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.945911 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.879384 24005 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:07.939285 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:07.949738 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:07.853641 24309 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:07.967607 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 263679us
I20260709 13:21:07.989244 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:07.990455 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:07.983585 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:07.986047 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:07.949847 16634 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:07.936184 24324 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:07.937160 24250 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:07.991808 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:07.965934 24005 raft_consensus.cc:3060] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.012084 17416 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:08.014946 16742 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
I20260709 13:21:07.997305 24270 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:07.982945 24263 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:08.016698 24270 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:08.022526 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 275748us
I20260709 13:21:08.020099 16740 raft_consensus.cc:1240] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:08.007591 24291 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:08.040844 16740 raft_consensus.cc:1240] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:08.041803 16669 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:08.041978 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.009573 23694 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:08.025400 24270 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:08.062207 17374 leader_election.cc:336] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.080996 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:08.079149 17374 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:08.084251 17374 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.038990 24267 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:08.097504 24055 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:08.103655 24330 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:21:08.103682 16745 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:08.108933 24208 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.054353 24269 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:08.117288 23489 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:08.122072 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 131713us
I20260709 13:21:08.122349 16669 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:08.125818 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:08.087906 24314 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:08.141253 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.138599 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:08.166353 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 216524us
I20260709 13:21:08.174520 16669 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 13:21:08.175284 16669 pending_rounds.cc:85] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 13
I20260709 13:21:08.175626 16669 pending_rounds.cc:107] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 13:21:08.175971 16669 raft_consensus.cc:2889] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.178462 23489 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:08.184841 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 46342.2us
I20260709 13:21:08.185887 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:08.064152 24056 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:08.185704 16634 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.193153 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.193638 23489 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.200788 23489 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:08.214895 16634 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.204952 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:08.238873 17374 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.240047 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:08.186749 24267 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:21:08.247583 16634 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:08.254487 16634 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.253269 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:08.229977 24133 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.002s
W20260709 13:21:08.297675 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 44466.7us
I20260709 13:21:08.301548 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:08.322750 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.322973 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 129919us
I20260709 13:21:08.323987 16636 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.325063 23489 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.325227 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.203480 24314 consensus_queue.cc:1059] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.099s
I20260709 13:21:08.251614 24291 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:08.218324 24152 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.216377 24321 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.110s
I20260709 13:21:08.230365 24336 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 13:21:08.206679 24224 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.134s
I20260709 13:21:08.213932 24060 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:08.251754 24267 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:08.210676 24159 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:08.337391 24053 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:08.352000 16666 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:21:08.352581 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:08.353741 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:08.380268 16666 pending_rounds.cc:85] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 12
I20260709 13:21:08.380739 16666 pending_rounds.cc:107] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:21:08.301764 24242 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:08.207369 24164 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.250587 23980 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:21:08.341590 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:08.363581 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:08.321658 24133 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.332543 23636 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:08.395704 16743 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:08.191612 23694 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:08.253705 24347 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 13:21:08.367162 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:08.407131 24055 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.407289 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:08.408550 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:08.259454 24315 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:08.411268 24315 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.411717 24315 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:08.415449 24332 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:21:08.351748 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:08.366222 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:08.416285 16667 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:21:08.416483 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 50356.7us
W20260709 13:21:08.355171 23811 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.533s	user 0.011s	sys 0.000s
I20260709 13:21:08.341234 24291 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.339957 24153 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:08.381116 16666 raft_consensus.cc:2889] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:08.385206 16664 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:21:08.363778 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:08.372687 24267 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:08.402479 24056 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.404075 16736 raft_consensus.cc:2393] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 15: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:08.398425 24289 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.420360 17416 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:08.419588 16559 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.389381 24273 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:08.391084 24320 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:08.454487 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.395740 24053 raft_consensus.cc:697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:08.425258 24055 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:21:08.473186 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 150505us
I20260709 13:21:08.476912 16559 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.478062 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.425702 24347 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:21:08.458643 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:08.425377 24153 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:08.426988 24291 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:08.462910 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:08.469447 16667 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:08.459865 24303 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:08.486070 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:08.458395 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 126564us
I20260709 13:21:08.495914 24321 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:08.419162 23811 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.457880 24073 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:21:08.476109 24058 raft_consensus.cc:3060] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:08.504189 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 122508us
W20260709 13:21:08.480934 24348 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:21:08.456463 24289 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.510192 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.460780 24263 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:08.472388 24055 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:08.507688 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:08.504756 24153 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:08.523377 16667 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:08.479825 24281 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:21:08.525686 24055 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:08.524279 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.537451 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 83007.4us
I20260709 13:21:08.476096 24273 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.535739 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:08.504402 24321 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:21:08.531277 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.516717 24060 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.548502 16743 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 21: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
W20260709 13:21:08.549373 16634 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:08.530828 24264 consensus_meta.cc:287] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.001s
W20260709 13:21:08.558429 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.558562 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:08.559254 17411 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:21:08.563274 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:08.512128 24282 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 13:21:08.536417 24303 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:08.563148 16738 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:21:08.565701 16708 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.564054 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:08.573561 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.574129 16743 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:08.574366 16708 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.574985 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.581645 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:08.582746 16739 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 22: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:08.530447 23980 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
W20260709 13:21:08.575372 24298 consensus_meta.cc:287] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.903s	user 0.011s	sys 0.001s
W20260709 13:21:08.600252 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.568799 24264 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:08.605877 16663 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:21:08.605098 16708 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:08.612650 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.618659 16667 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:08.622387 24164 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.610486 16663 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:08.625274 16663 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.626546 16663 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:08.627426 24242 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:08.627892 24242 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:21:08.628717 16708 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.629228 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.628887 24242 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.629966 24242 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:08.582273 24224 raft_consensus.cc:2804] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:08.585425 24073 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:08.593760 24279 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:08.597545 24282 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:21:08.648378 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.650127 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.635226 24005 consensus_meta.cc:287] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.613s	user 0.004s	sys 0.007s
I20260709 13:21:08.598582 24356 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:08.597831 24258 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:08.649634 17410 raft_consensus.cc:1275] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:08.652139 16744 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:08.654925 17411 raft_consensus.cc:2468] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 15.
I20260709 13:21:08.611268 24159 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:08.636752 24349 log.cc:927] Time spent T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
W20260709 13:21:08.644726 24324 consensus_meta.cc:287] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.629s	user 0.004s	sys 0.007s
I20260709 13:21:08.592626 24060 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
W20260709 13:21:08.657102 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.641652 24073 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:08.665346 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:08.632292 24361 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:08.643627 24133 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.581281 24053 consensus_queue.cc:237] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:08.651386 24362 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:08.677376 16708 leader_election.cc:336] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.657765 24264 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:08.651886 24224 raft_consensus.cc:697] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:08.654740 24282 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:08.678200 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.684583 24354 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:21:08.662091 24005 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:08.691331 24353 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:21:08.692363 24357 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:21:08.750314 24005 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.698565 24315 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.752718 24315 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:08.753499 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76205us
W20260709 13:21:08.754200 16708 leader_election.cc:336] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.754560 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.755694 24289 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:08.755681 16708 leader_election.cc:336] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.756650 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:08.741581 16710 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:08.704135 24273 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.639109 24321 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.656430 23811 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.659845 24235 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:08.670411 24373 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:08.722723 16745 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:08.676916 24351 log.cc:927] Time spent T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
W20260709 13:21:08.760040 16708 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.807771 16708 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:08.808589 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48570.9us
W20260709 13:21:08.809155 16708 leader_election.cc:336] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.809427 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:08.764621 24072 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:08.765606 24073 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:08.816540 24073 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:08.815948 24282 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.817643 24282 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:08.778687 24133 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:08.765865 24360 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:08.819283 24373 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.765969 23788 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:08.822410 24373 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.793430 24309 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.747659 24224 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.797783 16743 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
W20260709 13:21:08.811717 16708 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:08.865404 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53703.3us
W20260709 13:21:08.866326 16708 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.810660 24051 raft_consensus.cc:2804] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:21:08.815394 24235 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:08.879390 24267 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:08.880802 24310 consensus_meta.cc:287] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 1.041s	user 0.012s	sys 0.001s
I20260709 13:21:08.881058 16665 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:08.881337 24310 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:08.882191 24267 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:08.832983 24306 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:08.885483 24058 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:08.832496 16634 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.833766 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:08.791383 24358 consensus_queue.cc:1048] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:21:08.907588 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 75131us
I20260709 13:21:08.845685 16663 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:21:08.829566 24321 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:08.843232 24368 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.080s	user 0.002s	sys 0.002s
I20260709 13:21:08.790241 24055 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.912436 24055 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:08.866899 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.913595 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47315us
W20260709 13:21:08.914229 16708 leader_election.cc:336] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.914597 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.915630 16708 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.915988 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.916312 24270 consensus_meta.cc:287] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.829s	user 0.012s	sys 0.000s
I20260709 13:21:08.916744 24270 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:08.917183 16708 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.917603 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:08.920717 16708 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:08.927332 16708 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:08.923600 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 894a0fdf94344f46b0426efbd37a6c40 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ba2df6a668144022a223e504ae571f21 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:08.933056 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 408876us
I20260709 13:21:08.933851 17374 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:08.934815 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 427262us
I20260709 13:21:08.936012 16559 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:08.938721 16708 leader_election.cc:336] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.950364 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.947337 16559 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.934437 24308 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:08.949797 16634 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 121 similar messages]
I20260709 13:21:08.998407 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:08.950712 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:09.002431 16740 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:09.002482 23489 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:09.002729 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 52970.9us
W20260709 13:21:09.004519 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57257.9us
W20260709 13:21:08.985826 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 373313us
I20260709 13:21:08.973412 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:09.004122 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65448.3us
I20260709 13:21:08.918844 24270 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:09.004977 16634 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:08.934676 24133 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:09.008849 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.014132 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:09.010814 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:09.015002 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.011587 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:09.016579 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:08.983356 16636 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.010s	sys 0.005s
W20260709 13:21:09.019605 16636 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.012s	sys 0.005s
W20260709 13:21:09.020119 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 689677us
I20260709 13:21:09.022971 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:08.934777 24358 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:09.023902 16663 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:21:09.020156 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47016.5us
I20260709 13:21:08.949497 24058 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:21:08.937183 24314 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.000s	sys 0.011s
I20260709 13:21:08.951781 24051 raft_consensus.cc:697] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:09.030683 16670 raft_consensus.cc:3055] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:09.031438 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.033627 16670 raft_consensus.cc:740] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:08.985224 24379 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:08.938832 24243 consensus_meta.cc:287] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 1.119s	user 0.013s	sys 0.000s
I20260709 13:21:08.934615 24360 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:09.005455 24374 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:09.035132 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:09.038416 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:09.039342 16561 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:09.039499 16739 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:21:09.030453 24314 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.003s	sys 0.014s
I20260709 13:21:09.035511 24058 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:08.951339 24291 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:08.951412 24153 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:09.127878 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 88583.6us
I20260709 13:21:09.033883 24242 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.125304 16670 consensus_queue.cc:260] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.033840 24380 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:21:09.144002 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 110020us
W20260709 13:21:09.144918 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 107829us
I20260709 13:21:09.178493 16668 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:09.119004 24279 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:09.189443 16743 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:09.204257 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.206044 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.206414 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.149278 24380 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:09.156677 24289 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:09.143776 24310 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.225478 17414 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 13: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:09.037721 24373 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.111s
I20260709 13:21:09.230609 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.233124 17415 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:09.231379 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.225457 24380 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:09.208590 23636 consensus_queue.cc:1059] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:21:09.241062 23636 raft_consensus.cc:3055] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:09.241488 23636 raft_consensus.cc:740] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:21:09.243193 24386 log.cc:927] Time spent T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:21:09.187786 24242 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.125238 24243 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.255785 17420 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:09.260576 24153 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.261608 24243 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.263160 24291 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:09.254724 24242 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:09.264312 24383 log.cc:927] Time spent T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
W20260709 13:21:09.268666 24385 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:21:09.194237 24279 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:21:09.275887 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 241244us
I20260709 13:21:09.277350 16708 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
W20260709 13:21:09.278362 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 129532us
I20260709 13:21:09.279264 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.280073 17413 raft_consensus.cc:2368] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23.
I20260709 13:21:09.280994 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.281662 16741 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
W20260709 13:21:09.282768 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.283788 16559 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:09.181033 24053 consensus_queue.cc:1059] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 13:21:09.293960 24308 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:09.295607 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.307266 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.299939 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:09.316622 16741 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 22: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:09.316843 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.337039 17414 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:09.344432 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.297828 24133 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:09.317363 23811 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:09.354352 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.318099 16708 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:09.195971 24051 consensus_queue.cc:237] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.355907 23811 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.359115 23811 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.360709 24279 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.365757 24279 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.307695 24308 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:09.369905 24308 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:09.347069 24381 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:09.285892 23636 consensus_queue.cc:260] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.371433 24381 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.372589 23636 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:09.333004 24373 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:09.375070 24133 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:09.337316 24058 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:09.294097 24360 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:21:09.337848 24273 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.375213 24373 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:09.373833 24381 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:09.394796 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 168337us
W20260709 13:21:09.381090 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 63093.1us
I20260709 13:21:09.381171 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.387187 24373 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:09.386309 24364 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:09.397117 17414 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:09.399528 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.399688 16740 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:09.397953 24380 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.401872 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:09.401998 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.402839 24291 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:21:09.404078 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 213543us
I20260709 13:21:09.406910 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.410670 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.410730 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.420239 17414 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:09.422005 16740 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:09.403430 24291 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:09.431069 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.434451 17414 raft_consensus.cc:1240] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:09.412638 24321 consensus_meta.cc:287] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 13:21:09.438360 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.388131 24273 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.441121 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.446437 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.447703 16740 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:09.428717 24364 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:09.453058 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.437225 24380 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.409624 24264 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 13:21:09.476915 16634 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.444891 24306 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.457679 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:09.489356 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 205588us
I20260709 13:21:09.502264 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.507057 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:09.498214 24282 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.680s	user 0.012s	sys 0.000s
I20260709 13:21:09.510416 17414 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:09.513415 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.512068 16737 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 20: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:09.514375 16738 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:21:09.500681 24372 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:21:09.519833 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.515661 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:09.448153 24291 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.529873 17414 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:09.439083 24133 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:09.533490 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.518167 24250 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.506069 24058 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:21:09.501266 24306 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.515110 24289 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:09.551729 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.568306 16737 raft_consensus.cc:1275] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:21:09.575475 15476 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:09.575882 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.581404 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:09.580991 24392 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.598178 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.599177 17410 raft_consensus.cc:2368] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23.
I20260709 13:21:09.584558 24005 consensus_queue.cc:1059] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:21:09.605198 24005 consensus_queue.cc:1243] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 13:21:09.605587 24395 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:21:09.605788 24005 raft_consensus.cc:3055] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:21:09.606366 24005 raft_consensus.cc:740] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:09.561514 24364 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.587332 24370 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
W20260709 13:21:09.615437 24401 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:21:09.619832 24273 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.619984 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.621107 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.621373 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.621811 17410 raft_consensus.cc:2468] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 14.
I20260709 13:21:09.621899 17415 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:09.622864 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.625566 24289 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.626856 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.627578 17409 raft_consensus.cc:2468] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:09.628880 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.587608 24381 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:21:09.598055 24392 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.630388 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.631541 24392 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.633241 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.633963 17410 raft_consensus.cc:2468] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:09.640698 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:09.644081 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.646077 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:09.650451 16738 raft_consensus.cc:2468] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
I20260709 13:21:09.651124 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:09.654799 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.564344 24250 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.645171 24005 consensus_queue.cc:260] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.661664 24005 raft_consensus.cc:3060] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:09.665954 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:09.656344 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:09.695604 16634 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:09.699317 16634 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.649816 24152 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:09.705050 17410 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
W20260709 13:21:09.707573 16708 leader_election.cc:336] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.648730 24392 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:21:09.712050 24392 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:09.712636 24392 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:09.713495 24392 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.657436 24235 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:09.716902 24235 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.720300 24320 raft_consensus.cc:3055] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:09.720891 24320 raft_consensus.cc:740] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:09.708076 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:09.726173 16708 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:09.727064 16708 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.731235 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:09.732887 16708 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:09.734153 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:09.737977 16708 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.768599 17419 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
W20260709 13:21:09.779868 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 49954.2us
I20260709 13:21:09.783347 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.722080 24320 consensus_queue.cc:260] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.791486 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.791885 24320 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:09.792102 24250 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.793396 24235 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.789141 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.789914 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.791208 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.714787 24392 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:09.799439 17409 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 14. Current term is 15. Ops: []
W20260709 13:21:09.797729 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:09.800338 16634 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.803344 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.804164 17414 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:09.804906 24378 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:09.794067 24270 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.807441 24270 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.783115 24364 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:21:09.810928 24404 log.cc:927] Time spent T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:21:09.811901 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.819574 24306 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:09.821347 24308 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.824766 16741 raft_consensus.cc:1240] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:09.795580 24407 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:09.827567 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.810537 24270 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.791524 24373 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.835736 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.824708 24308 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:09.834443 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:09.833329 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:09.837179 17419 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:09.799602 24152 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.838186 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.837320 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:09.837478 16708 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.839926 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:09.845266 16708 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.849187 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:09.855885 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:09.862416 16708 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.863907 24373 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.864780 17415 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:21:09.871172 24308 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:09.884650 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:09.885768 16708 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.889145 16741 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:09.847234 24152 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.896145 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.896886 17415 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:09.903671 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:09.905735 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.907332 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:09.913996 24388 log.cc:927] Time spent T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.580s	user 0.000s	sys 0.015s
I20260709 13:21:09.916680 24152 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:09.940604 16636 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.918009 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.921828 24392 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.941871 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:09.942255 16741 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:09.923308 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:09.924904 24073 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.946487 24073 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:09.930457 17420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48626 (request call id 2247) took 1098 ms. Trace:
W20260709 13:21:09.933738 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:09.950240 17420 rpcz_store.cc:276] 0709 13:21:08.831955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.833515 (+  1560us) service_pool.cc:224] Handling call
0709 13:21:09.930204 (+1096689us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400465354. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe82fa 00007f3284ba9b17 00007f327e1246d4 00007f327e124662 00007f327e124591 00007f327e12450c 00007f327e1237c2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:09.930355 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799970560}
I20260709 13:21:09.951148 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.952104 16741 raft_consensus.cc:2468] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
W20260709 13:21:09.954453 16634 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:09.956269 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica efa00eb5e7404570958fdebb6cd077e8 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:09.957409 24073 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:09.942734 16666 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:09.961126 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.962668 17420 raft_consensus.cc:2410] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:21:09.965004 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:09.963832 24058 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:09.967520 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:09.968463 16671 raft_consensus.cc:2410] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:21:09.967348 24058 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.969594 24308 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:09.971335 24058 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.970911 24392 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:09.973433 24392 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:09.974074 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:09.977380 16742 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2915) took 1532 ms. Trace:
I20260709 13:21:09.977870 16742 rpcz_store.cc:276] 0709 13:21:08.444687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.446593 (+  1906us) service_pool.cc:224] Handling call
0709 13:21:08.468283 (+ 21690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:08.474759 (+  6476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:08.474804 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:08.499950 (+ 25146us) log.cc:844] Serialized 26 byte log entry
0709 13:21:08.802131 (+302181us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b1800114040. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:21:09.020806 (+218675us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:09.020847 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.020922 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:09.913404 (+892482us) spinlock_profiling.cc:243] Waited 1.42 ms on lock 0x7b4400022b4c. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327ca1dc85 00007f327ca1e6a2 00007f327f7901db 00007f327f78baaf 00007f327f77f86e 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221 00007f327f7881d9 00007f327f788169
0709 13:21:09.977161 (+ 63757us) raft_consensus.cc:1685] finished
0709 13:21:09.977183 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:09.977314 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8724,"raft.queue_time_us":6039,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5133,"spinlock_wait_cycles":6912640,"thread_start_us":218453,"threads_started":1,"wal-append.queue_time_us":833316}
I20260709 13:21:09.982235 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3477) took 1637 ms. Trace:
I20260709 13:21:09.982722 16668 rpcz_store.cc:276] 0709 13:21:08.344709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:08.461417 (+116708us) service_pool.cc:224] Handling call
0709 13:21:09.147168 (+685751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.187789 (+ 40621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:09.982068 (+794279us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b540009e050. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe38ad 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6 00007f327df1e26f
0709 13:21:09.982177 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1544329088}
I20260709 13:21:09.988329 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:09.993654 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:09.998993 16634 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:09.998713 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.003571 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.005722 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.005697 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.006597 17414 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:10.015468 16636 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:10.008955 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:10.019789 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:10.023490 24359 consensus_queue.cc:1048] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:10.017684 24310 consensus_queue.cc:1059] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:21:10.024772 24056 consensus_queue.cc:1059] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 13:21:10.025220 24250 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:10.024349 24152 raft_consensus.cc:2804] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:10.027196 24152 raft_consensus.cc:697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:10.028263 24152 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.030303 24005 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:10.030831 24005 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.031244 24005 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:10.033617 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.038698 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.054508 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.058501 24310 raft_consensus.cc:3055] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:10.059000 24310 raft_consensus.cc:740] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:10.059867 24310 consensus_queue.cc:260] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.061133 24310 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:10.063201 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.063913 17411 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:10.064431 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.064630 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.065466 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:10.065928 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.066825 17410 raft_consensus.cc:2393] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:10.067193 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.067559 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.068331 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.068703 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.068929 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.069116 16669 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 21: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:10.069725 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.069832 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.069967 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.071152 24056 raft_consensus.cc:3055] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:10.071214 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.077126 16710 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.065534 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:10.074402 16637 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.017959 24343 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:21:10.085271 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:10.099252 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79587.1us
I20260709 13:21:10.086022 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:10.072286 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.066015 17419 raft_consensus.cc:2376] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24. Re-sending same reply.
I20260709 13:21:10.098729 16666 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:10.101912 16559 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.104542 17420 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:21:10.104856 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.099753 16664 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:21:10.105530 16666 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:10.106285 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.039849 23788 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:10.108706 16669 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 20: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:10.120332 24060 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:10.120674 24172 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:10.120908 16559 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.121392 24172 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:10.121732 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.123608 23980 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:10.072463 24308 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:10.123720 16559 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:10.125290 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48268.4us
I20260709 13:21:10.126226 16736 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:10.126446 16670 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:10.114038 24381 consensus_queue.cc:1059] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:10.126629 16710 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.125314 24406 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:10.131728 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.075716 24053 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:10.133665 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.113320 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.139874 16667 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:21:10.090487 24056 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:10.142180 16668 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:21:10.098392 24058 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.137574 24396 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:10.145298 16666 raft_consensus.cc:2468] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:21:10.148098 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.148842 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.150926 16667 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:21:10.151353 16667 pending_rounds.cc:85] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 15
I20260709 13:21:10.151600 16667 pending_rounds.cc:107] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:21:10.151834 16667 raft_consensus.cc:2889] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:10.107232 24055 raft_consensus.cc:2804] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:10.148802 16669 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:10.160831 16559 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:10.168450 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.165096 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.171039 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.173120 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.186060 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:10.160771 17374 connection.cc:256] RPC call timeout handler was delayed by -1.1088220520000505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 13:21:10.122531 24172 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:10.194499 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:10.195334 16742 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:10.202206 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.206581 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.208403 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.188458 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:10.144997 24391 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:10.193219 23636 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:10.210354 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.161573 24055 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.161751 24347 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.168009 24406 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:10.223038 16635 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.164287 24396 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:10.187538 24359 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:10.161375 24058 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.184994 24397 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:10.215796 24053 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:21:10.290638 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 148505us
W20260709 13:21:10.293963 24420 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:21:10.296554 24005 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:10.224710 24224 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:10.221024 24373 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:10.299690 24373 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:10.300154 24373 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:10.300364 24172 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:10.299791 24396 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:10.219921 24055 raft_consensus.cc:3060] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:10.290041 24056 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.305025 24056 raft_consensus.cc:3060] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:10.313156 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.232125 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.186930 24270 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:10.214457 24279 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:10.280889 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 146368us
I20260709 13:21:10.283803 16667 raft_consensus.cc:1240] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
W20260709 13:21:10.314931 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 82968.8us
I20260709 13:21:10.213565 24381 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:21:10.329630 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:10.330356 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.325646 16742 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
W20260709 13:21:10.341300 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.272711 24392 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:10.290354 24397 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.346706 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:10.357636 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:10.359359 16662 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:10.358783 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 181499us
I20260709 13:21:10.331353 24267 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:10.368199 24005 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:10.369314 24005 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:10.369465 24309 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:10.370030 24309 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:10.369994 24005 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:10.363221 23980 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:21:10.230705 24347 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:10.327868 24053 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:10.381464 24309 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:10.363288 24397 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:10.386834 24053 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.382987 24398 raft_consensus.cc:3055] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:10.383704 24058 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:10.384990 16669 raft_consensus.cc:1275] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:10.386346 16666 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:10.397215 16669 raft_consensus.cc:1240] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 13:21:10.391868 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:10.398270 16743 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:10.396726 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.389025 24053 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:10.393208 16662 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
W20260709 13:21:10.410486 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.412176 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.414173 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.418747 17412 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:10.420855 17412 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:10.422482 16669 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:10.399961 24398 raft_consensus.cc:740] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:10.412941 24152 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:10.433207 16669 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:21:10.429267 17374 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:10.414826 24320 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:10.437969 17412 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:10.412334 24310 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:21:10.454144 16670 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:21:10.456714 16670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3484) took 1426 ms. Trace:
W20260709 13:21:10.458614 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 108700us
I20260709 13:21:10.462605 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:10.460945 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.467657 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:10.472177 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.419452 24055 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.457558 16670 rpcz_store.cc:276] 0709 13:21:09.030071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.030225 (+   154us) service_pool.cc:224] Handling call
0709 13:21:09.030518 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:09.824414 (+793896us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b54000cd8a0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb59a1d 00007f327fb401a4 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419
0709 13:21:10.126017 (+301603us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b580096d7d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:21:10.456334 (+330317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.456650 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1077681280}
I20260709 13:21:10.466813 16669 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:21:10.528878 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 66321.6us
I20260709 13:21:10.529892 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:10.530100 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 183476us
I20260709 13:21:10.467456 24320 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:21:10.576048 15476 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:10.537803 24358 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:10.635075 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 105274us
W20260709 13:21:10.615244 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 178618us
I20260709 13:21:10.546167 24340 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.640437 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.641009 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:10.547260 24417 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:21:10.632364 16708 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.641681 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.547747 24320 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:10.648000 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.656117 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.659022 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.660923 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.661782 17415 raft_consensus.cc:2368] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23.
I20260709 13:21:10.663079 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.664866 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.657380 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.633386 24310 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:10.684593 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:10.685505 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 48640.6us
I20260709 13:21:10.686335 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.688287 16561 leader_election.cc:400] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23.
I20260709 13:21:10.693061 16636 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.650664 24320 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:10.653074 24060 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:10.660393 24133 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.668800 24279 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:10.647324 24358 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.675324 24053 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:10.733626 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 44113.4us
I20260709 13:21:10.668190 24436 raft_consensus.cc:2804] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:10.747771 24453 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:10.694571 24270 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:10.756603 24435 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:10.757624 24435 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.735605 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:10.761158 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:10.762385 16561 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:10.763671 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:10.764880 24444 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:10.765549 24444 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:10.766156 24444 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:10.767194 24453 raft_consensus.cc:2804] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:21:10.767685 24453 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:10.768098 24453 raft_consensus.cc:3060] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:21:10.768981 24454 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:10.649928 24340 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.774799 16670 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:10.776263 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:10.776911 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:10.777107 16670 raft_consensus.cc:1240] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:10.778642 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:10.737810 24451 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:10.738008 24310 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:10.793637 24398 consensus_queue.cc:260] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.793170 16489 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:10.704732 24421 log.cc:927] Time spent T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
I20260709 13:21:10.739733 24056 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:10.799657 16489 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:10.801630 24056 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:10.802079 24056 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:10.802464 24056 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:10.731645 24060 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:10.802642 16489 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 16 to 18, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:10.730712 24133 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:10.741376 24439 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:21:10.743016 24441 log.cc:927] Time spent T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:21:10.705108 24445 consensus_queue.cc:1048] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
W20260709 13:21:10.840868 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 77275.2us
I20260709 13:21:10.842028 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:10.743969 24433 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 13:21:10.751334 24426 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 13:21:10.754047 17414 raft_consensus.cc:1275] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:10.691684 24378 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:10.794363 24310 raft_consensus.cc:2804] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:21:10.742268 24419 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:21:10.804877 16663 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3483) took 1776 ms. Trace:
W20260709 13:21:10.812464 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 119506us
I20260709 13:21:10.854156 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.859735 16663 rpcz_store.cc:276] 0709 13:21:09.028261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.028549 (+   288us) service_pool.cc:224] Handling call
0709 13:21:09.028930 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:09.032772 (+  3842us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:09.032802 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:09.064628 (+ 31826us) log.cc:844] Serialized 26 byte log entry
0709 13:21:09.985632 (+921004us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b18001ab0c0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:21:10.143247 (+157615us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:21:10.143288 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.143630 (+   342us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:10.804556 (+660926us) raft_consensus.cc:1685] finished
0709 13:21:10.804577 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:10.804815 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3071,"raft.queue_time_us":4127,"raft.run_cpu_time_us":8130,"raft.run_wall_time_us":151811,"spinlock_wait_cycles":3100416,"thread_start_us":157414,"threads_started":1,"wal-append.queue_time_us":1105782}
I20260709 13:21:10.861604 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
W20260709 13:21:10.843636 24428 log.cc:927] Time spent T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.514s	user 0.001s	sys 0.003s
I20260709 13:21:10.868126 24435 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:10.869052 24427 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.334s	user 0.003s	sys 0.000s
I20260709 13:21:10.869980 24073 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:10.872350 16636 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:21:10.872845 16636 leader_election.cc:403] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:21:10.868841 24435 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:10.875677 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:10.804428 24358 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:10.810456 24269 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:10.796942 24055 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:10.807152 24133 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:10.876459 17415 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 24.
I20260709 13:21:10.802805 23980 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:21:10.840903 24373 consensus_meta.cc:287] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.004s
I20260709 13:21:10.881213 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.901912 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:10.855471 24398 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:10.844391 24279 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:10.804867 24060 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:10.862360 24310 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:10.878448 24224 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:10.990121 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.884275 24359 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
W20260709 13:21:11.029173 17413 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.021s	sys 0.040s
I20260709 13:21:10.920890 24358 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:10.936823 24306 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:21:11.127247 17413 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.021s	sys 0.041s
W20260709 13:21:11.127939 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:11.035128 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:10.915447 24378 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:10.921283 24055 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:11.086375 16671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3485) took 1101 ms. Trace:
W20260709 13:21:11.048607 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:11.131168 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 351071us
W20260709 13:21:11.132357 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 270753us
W20260709 13:21:11.125872 24309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.016s	sys 0.017s
I20260709 13:21:10.920531 24270 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:11.153115 17376 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.017s	sys 0.046s
W20260709 13:21:11.107882 24397 consensus_meta.cc:287] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.003s
I20260709 13:21:10.995786 24310 raft_consensus.cc:3060] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:11.004537 24456 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:21:11.025005 23980 raft_consensus.cc:2804] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:11.087989 24464 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:11.163645 17376 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.018s	sys 0.047s
I20260709 13:21:11.152452 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:11.177518 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:11.166239 16668 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.017s	sys 0.021s
I20260709 13:21:10.915997 24373 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.126087 24436 raft_consensus.cc:697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:11.129348 24370 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.147047 16671 rpcz_store.cc:276] 0709 13:21:09.984599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:09.984806 (+   207us) service_pool.cc:224] Handling call
0709 13:21:10.013030 (+ 28224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:10.015335 (+  2305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:10.015367 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:10.015844 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 13:21:10.112164 (+ 96320us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b18002f48e0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:21:10.311321 (+199157us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:21:10.311363 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:10.311436 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:11.081800 (+770364us) raft_consensus.cc:1685] finished
0709 13:21:11.081825 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.086271 (+  4446us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1818,"raft.queue_time_us":3579,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":7480,"spinlock_wait_cycles":9852160,"thread_start_us":198952,"threads_started":1,"wal-append.queue_time_us":398829}
I20260709 13:21:11.129524 24435 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.135583 23636 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:11.156405 24309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.016s	sys 0.017s
I20260709 13:21:10.936877 24235 consensus_queue.cc:1048] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.220s
W20260709 13:21:11.178534 16668 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.017s	sys 0.021s
I20260709 13:21:11.036760 24258 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:10.996172 24224 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:11.179872 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 260236us
I20260709 13:21:11.198042 16668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3499) took 1132 ms. Trace:
I20260709 13:21:11.140573 24306 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.093259 24444 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.176322 24055 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.174458 24340 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.174731 23980 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:11.212442 24152 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.212205 24470 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:11.145864 24358 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.215207 24060 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.202597 24343 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:11.220567 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.175490 24397 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.235867 24454 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:11.236382 24397 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.218286 24250 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.222038 23980 raft_consensus.cc:3060] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:11.219218 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 281608us
I20260709 13:21:11.207391 24258 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.181283 24464 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.216459 24374 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.270571 23980 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.233492 16736 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:11.223697 24470 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.334410 24470 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:11.335984 24443 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.800s	user 0.000s	sys 0.002s
W20260709 13:21:11.339226 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 77556.9us
I20260709 13:21:11.214402 16668 rpcz_store.cc:276] 0709 13:21:10.065744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.153204 (+ 87460us) service_pool.cc:224] Handling call
0709 13:21:10.153489 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.153722 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:21:10.443400 (+289678us) log.cc:844] Serialized 14 byte log entry
0709 13:21:10.810185 (+366785us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b1800479da0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f3283ab858f 00007f327caf7571 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd
0709 13:21:11.197079 (+386894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:11.197293 (+   214us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:21:11.197498 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.197772 (+   274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.197960 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4174,"prepare.queue_time_us":467413,"prepare.run_wall_time_us":62,"raft.queue_time_us":100198,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":11780,"spinlock_wait_cycles":125870336,"thread_start_us":1028376,"threads_started":3,"wal-append.queue_time_us":224445}
I20260709 13:21:11.341358 16489 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 20 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:11.342648 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.343494 17420 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:11.343174 16489 catalog_manager.cc:5697] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 13 to 18, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:11.344673 16670 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:21:11.344905 16489 catalog_manager.cc:5697] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:11.345661 24267 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.195858 24370 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.346200 24267 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.249003 24358 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:11.222606 24309 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.347489 24309 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.223939 24152 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:11.348577 24370 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.348480 24267 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.351719 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36074 (request call id 3500) took 1217 ms. Trace:
W20260709 13:21:11.252770 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 91670.4us
I20260709 13:21:11.253703 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.253592 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.352209 16666 rpcz_store.cc:276] 0709 13:21:10.134651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.410934 (+276283us) service_pool.cc:224] Handling call
0709 13:21:10.411277 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.411492 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:21:10.907624 (+496132us) log.cc:844] Serialized 14 byte log entry
0709 13:21:11.351327 (+443703us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:11.351383 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:11.351440 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.351518 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.351659 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":198156,"prepare.queue_time_us":669591,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":6256,"raft.queue_time_us":198690,"raft.run_cpu_time_us":62,"raft.run_wall_time_us":62,"thread_start_us":728234,"threads_started":2,"wal-append.queue_time_us":415336}
I20260709 13:21:11.354130 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.346351 16489 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 21 to 22, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 22 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:11.355895 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.356886 16670 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:11.350482 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.359306 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.218731 24468 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.231652 17413 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 2267) took 1042 ms. Trace:
I20260709 13:21:11.237280 24250 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.262517 24467 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:11.390295 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:11.264482 24306 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.268293 24258 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.202924 24435 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:11.392684 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66040.5us
I20260709 13:21:11.393405 24435 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.320082 24471 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:11.394182 24471 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.396613 24471 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:11.236867 24453 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.322376 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:11.322907 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:11.323205 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:11.323814 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:11.324570 23980 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.324618 24224 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.246753 24436 consensus_queue.cc:237] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.404426 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:11.405107 17411 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:11.327710 24444 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.352209 24309 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.223634 24055 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.329475 24464 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.330272 24378 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:11.330129 24374 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.322582 24373 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:11.253384 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.255726 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.379691 16667 raft_consensus.cc:2393] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 26: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:11.417037 16664 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:11.382570 16708 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.390697 24468 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:11.398996 17374 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.401114 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:11.398578 17416 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 18.
I20260709 13:21:11.327883 24056 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:11.388412 24320 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.685s	user 0.005s	sys 0.008s
I20260709 13:21:11.398789 24133 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.418005 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:11.378015 17413 rpcz_store.cc:276] 0709 13:21:10.189677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.189872 (+   195us) service_pool.cc:224] Handling call
0709 13:21:10.190195 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:10.190419 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:10.354494 (+164075us) log.cc:844] Serialized 14 byte log entry
0709 13:21:11.128376 (+773882us) log.cc:844] Serialized 14 byte log entry
0709 13:21:11.229803 (+101427us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:11.229900 (+    97us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:11.229973 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:11.230091 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:11.231563 (+  1472us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":180837,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":96233,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":1150,"spinlock_wait_cycles":14401408,"thread_start_us":964816,"threads_started":3}
I20260709 13:21:11.421999 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:11.423599 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.424309 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:11.411123 24306 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:11.407464 24451 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
W20260709 13:21:11.432157 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:11.433835 23788 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:11.411819 24310 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.429471 24306 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.464131 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:11.461561 16634 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.432188 24133 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.514205 24133 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:11.449586 24475 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:11.463729 24468 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.446662 24060 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.516122 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:11.429344 24320 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.464434 24306 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:11.497977 23788 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.510494 24474 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.510807 24250 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.498723 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:11.539366 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76288.2us
W20260709 13:21:11.539677 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 78160.9us
I20260709 13:21:11.540269 16561 leader_election.cc:400] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23.
W20260709 13:21:11.542774 16634 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:11.501929 24456 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:11.521657 24060 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:11.575074 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 168462us
I20260709 13:21:11.565701 23788 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.588897 24455 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:11.590456 23788 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:11.566039 24468 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:11.576845 16710 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:11.602077 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.602602 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.604132 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.606935 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.608924 16666 raft_consensus.cc:1240] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 13:21:11.611039 24289 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.611375 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.611630 24289 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.613976 24289 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.618161 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.619483 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:11.623872 24491 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.624605 24491 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.630376 24263 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.630486 24396 raft_consensus.cc:3055] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:21:11.631119 24396 raft_consensus.cc:740] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:11.631160 24263 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.632206 24396 consensus_queue.cc:260] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:11.636955 15476 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:11.638195 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:11.639250 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 240282us
I20260709 13:21:11.639839 16669 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:11.640268 24309 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:11.641089 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.641275 24445 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:11.664562 24445 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.665411 24235 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.667701 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:11.668320 16745 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.666182 24235 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.672241 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.659490 23788 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:11.672859 16743 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:11.643105 24320 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:11.596304 24398 consensus_meta.cc:287] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.676s	user 0.011s	sys 0.003s
I20260709 13:21:11.642490 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.675143 16666 raft_consensus.cc:2468] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 19.
I20260709 13:21:11.675922 24470 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.677033 24235 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.644127 24459 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:11.647030 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 47695.5us
I20260709 13:21:11.678550 24470 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:11.655290 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.547299 24474 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.597945 24453 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.672049 24488 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:21:11.672912 23788 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:21:11.644994 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.507678 24378 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.641748 24055 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:11.662067 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.681592 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:11.652426 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 54866.3us
I20260709 13:21:11.685268 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:11.686478 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:11.687352 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.714082 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.691464 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.700415 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.701154 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.700822 16745 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:11.703434 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:11.710294 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48269.2us
I20260709 13:21:11.690321 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.716235 16736 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:11.696673 24152 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.680047 24493 consensus_queue.cc:1059] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:21:11.718744 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.758931 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:11.719851 16743 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:21:11.720018 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.720018 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.726249 16664 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:11.726931 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:11.730494 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:11.711359 24472 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:11.734972 24491 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.739482 24133 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:11.741936 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.743520 16740 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 23. Ops: []
I20260709 13:21:11.745236 24474 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.746083 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.746306 24374 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.708029 23788 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:11.764166 16736 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:21:11.784735 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:11.765868 17413 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:11.766433 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:11.787667 17419 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 13.
I20260709 13:21:11.751101 24445 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.766973 17416 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:21:11.770206 16667 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:11.762598 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:11.792542 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 72576.6us
W20260709 13:21:11.793222 17374 leader_election.cc:336] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.779277 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:11.780118 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:11.794269 17374 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.794847 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:11.761237 24310 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.797441 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.791353 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:11.786819 24452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.029s	sys 0.050s
W20260709 13:21:11.806219 17374 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.695458 24459 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:11.811967 17374 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.803674 16736 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:11.777017 24133 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.813894 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.815898 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:11.821930 23636 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:11.822999 17409 raft_consensus.cc:1240] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23. Ops: []
W20260709 13:21:11.810660 24452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.030s	sys 0.050s
I20260709 13:21:11.775331 24472 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.826164 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:11.826757 17374 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.796660 24478 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:11.827914 17374 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.829809 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:11.827769 16710 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:11.815208 24474 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:21:11.832383 17374 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.828280 24451 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:11.816475 24133 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.816890 24320 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:11.839327 17374 leader_election.cc:336] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.839840 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:11.841166 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.842144 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:11.845523 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.845988 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:11.849834 24360 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 13:21:11.859778 24455 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:11.860783 24458 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:11.814630 24445 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:21:11.863924 16671 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:11.833559 24343 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:11.836136 24374 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:11.836911 24474 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.872102 16667 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:11.873019 24447 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:21:11.874050 24447 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:11.874408 24447 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:11.862473 24373 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:11.851886 24436 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:11.881428 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:11.835813 24164 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:21:11.867280 24343 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.871729 24374 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:11.890367 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.892717 16669 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:21:11.866554 24480 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:11.893817 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.893987 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.891381 16559 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:11.897583 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.905536 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:11.892832 16671 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:11.868286 24060 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.870939 24451 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.877249 24474 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.878988 24453 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.915623 23636 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:11.912048 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:11.916286 23636 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:11.910410 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:11.912712 24476 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.916385 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:11.920408 17412 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:11.903796 24343 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:11.903990 24374 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:11.952527 23636 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.954351 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.919565 24235 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:11.895167 24306 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.958312 24306 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.959088 24392 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:11.959671 24392 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.962081 24392 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.920692 24453 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.964596 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.965828 23636 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:11.966373 23636 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:11.968659 23636 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.964614 24453 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.975721 24152 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.978749 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:11.979532 17416 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:11.982882 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.917955 24258 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 13:21:11.984869 24492 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:11.985395 24492 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:11.915593 24267 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:11.986722 17414 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:11.903967 24381 consensus_queue.cc:1059] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:21:11.988278 24381 raft_consensus.cc:3055] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:11.988685 24381 raft_consensus.cc:740] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:11.989334 24347 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:11.989508 24381 consensus_queue.cc:260] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.999728 24056 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.026038 24432 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:12.026933 24432 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:12.020612 24381 raft_consensus.cc:3060] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:12.036150 24472 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:12.003273 24499 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:21:12.040938 24263 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:11.987227 24267 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.042621 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.044488 24267 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.046828 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:11.901985 24459 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:11.914655 24454 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:12.055948 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.056054 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:12.058449 16743 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:12.061925 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.064106 17416 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 25.
I20260709 13:21:12.085583 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.086905 16742 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:12.091372 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.066784 16710 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:12.108409 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.113749 16738 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 16.
I20260709 13:21:12.056865 24164 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:12.090633 24454 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:12.128178 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.107246 24306 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:12.141834 24454 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.137161 24267 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.145304 24267 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.147835 24267 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.153936 24396 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:12.159488 24375 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:12.182273 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 894a0fdf94344f46b0426efbd37a6c40 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ba2df6a668144022a223e504ae571f21 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:12.184103 17412 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 20 index: 17. (term mismatch)
I20260709 13:21:12.185249 17412 pending_rounds.cc:85] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 16
I20260709 13:21:12.138924 23788 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.188926 16741 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:12.185715 17412 pending_rounds.cc:107] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 19.17
I20260709 13:21:12.190689 17412 raft_consensus.cc:2889] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:12.124596 24164 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:12.198513 16559 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.208668 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.210024 16665 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:12.215724 17412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 2270) took 1816 ms. Trace:
I20260709 13:21:12.218398 16743 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:12.217487 17412 rpcz_store.cc:276] 0709 13:21:10.399515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:10.399644 (+   129us) service_pool.cc:224] Handling call
0709 13:21:10.418627 (+ 18983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:11.919878 (+1501251us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b58007748d0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fb400aa 00007f327fbc2447 00007f327fbdafe7 00007f327fbdc7de 00007f327fbe6b49 00007f327fbe7416 00007f327fbe3d8f 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4
0709 13:21:12.215445 (+295567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.215667 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378116864}
I20260709 13:21:12.204303 23788 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.207484 24392 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:12.217046 24320 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.223690 24378 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.241137 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:12.241863 24392 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.242059 24378 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.261214 24378 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.239308 24320 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.263468 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:12.267940 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 422426us
W20260709 13:21:12.268877 17374 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:12.269950 17374 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:11.875257 24447 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:12.271335 24501 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 13:21:12.285143 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.258492 24392 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.290588 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:12.292609 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:12.294013 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50777.2us
I20260709 13:21:12.297799 16738 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:12.271329 24447 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:21:12.300849 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.302095 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.299430 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.307329 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.311522 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.247825 24456 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:21:12.294096 24056 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:12.314343 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:12.295472 24172 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:12.296314 24373 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:12.297935 24310 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.319504 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:12.318795 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.319571 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.322197 17416 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:12.310700 24073 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:12.313369 24465 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:12.323194 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:12.326264 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:12.337680 16708 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.328889 17416 raft_consensus.cc:2468] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:12.315133 24055 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:12.338505 16708 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.339787 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.340277 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:12.316840 24370 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.012835 24476 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:11.915978 24451 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.160157 24375 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.322045 24491 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:12.321758 24056 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:12.329800 24310 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.329500 24470 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:12.326104 24306 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:21:12.354199 16708 leader_election.cc:336] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.343245 24381 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:12.352623 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:12.028317 24432 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.361025 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:12.360760 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85771.6us
I20260709 13:21:11.986334 24492 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.460060 23788 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:12.460165 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 107673us
I20260709 13:21:12.333501 24133 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.361390 24263 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.358412 24235 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:12.534953 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.462659 24056 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.463745 24381 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:12.465238 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.353109 24454 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:12.539757 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.536033 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:12.536577 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 170170us
I20260709 13:21:12.540441 17414 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.538480 17413 raft_consensus.cc:1240] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:21:12.538971 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.544483 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.548265 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:12.530467 24484 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:21:12.572731 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.573383 17412 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:12.550271 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:12.550376 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85248.9us
I20260709 13:21:12.576122 16668 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.576390 16561 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:12.550871 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.577215 16663 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:21:12.549803 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.552701 16662 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:12.578027 16740 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:12.578217 16664 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 15.
I20260709 13:21:12.359951 24491 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:12.580120 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.554279 16666 raft_consensus.cc:1275] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:12.555763 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:12.465339 24258 consensus_queue.cc:1175] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ba2df6a668144022a223e504ae571f21" responder_term: 20 status { last_received { term: 19 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:21:12.557902 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.558095 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:12.587337 17420 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:21:12.533239 24470 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.588281 17412 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:12.562682 17374 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.562495 24459 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:12.368314 24374 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.568432 24306 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.569545 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.533609 24343 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:12.579730 24474 consensus_meta.cc:287] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.000s
I20260709 13:21:12.551823 24381 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.589829 24055 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.580317 24347 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:12.591423 24060 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:12.540087 24263 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.604164 16668 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:12.549790 24375 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.614490 17414 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:12.615005 24055 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.616379 16670 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:12.615805 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53238.6us
I20260709 13:21:12.610661 24381 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:12.627262 24476 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.597600 24470 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.617532 24060 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.619787 24224 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.610088 24474 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.625104 24456 consensus_queue.cc:1059] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.015s
W20260709 13:21:12.661293 15476 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:12.634321 24491 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:12.664551 24515 log.cc:927] Time spent T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:21:12.664786 24374 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.670392 17420 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
W20260709 13:21:12.691105 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 97620.7us
I20260709 13:21:12.666671 24310 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.692871 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.671959 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.687238 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:12.696230 16743 raft_consensus.cc:3060] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:12.703955 24310 raft_consensus.cc:3060] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:12.690231 24133 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:12.708631 24055 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.685585 24060 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:12.721028 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 49206.9us
I20260709 13:21:12.721709 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:12.723433 16743 raft_consensus.cc:2468] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:12.688046 24343 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.728739 16636 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:12.695346 24451 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:12.751577 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:12.728129 24224 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.739107 24279 raft_consensus.cc:2804] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:12.739188 24459 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:12.751227 24374 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:12.826619 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:12.827039 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.827536 16738 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 23: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:12.813635 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.819558 16667 raft_consensus.cc:1240] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:12.770391 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:12.765856 16741 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2948) took 1085 ms. Trace:
I20260709 13:21:12.834300 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.832875 16741 rpcz_store.cc:276] 0709 13:21:11.680814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.684997 (+  4183us) service_pool.cc:224] Handling call
0709 13:21:11.685298 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.189425 (+504127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.765551 (+576126us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400433d54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:12.765778 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157682432}
W20260709 13:21:12.840725 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.878088 16743 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
W20260709 13:21:12.881386 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 188308us
I20260709 13:21:12.882644 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:12.809527 24518 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:21:12.892994 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:12.810395 24495 log.cc:927] Time spent T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.948s	user 0.007s	sys 0.004s
I20260709 13:21:12.760560 24373 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:21:12.768855 24474 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.797101 24508 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.814636 24511 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.922050 24511 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.767549 24406 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:12.923192 24406 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.924576 24511 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.790467 24435 raft_consensus.cc:3055] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:12.823637 24320 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.938194 24465 consensus_queue.cc:1059] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:12.938758 24463 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.939383 24463 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.938325 24435 raft_consensus.cc:740] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:12.941103 24435 consensus_queue.cc:260] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.941772 24463 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:12.896441 24505 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.660s	user 0.004s	sys 0.000s
I20260709 13:21:12.945355 16662 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:12.780380 24056 raft_consensus.cc:3055] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:21:12.778374 24396 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.624s	user 0.012s	sys 0.000s
I20260709 13:21:12.767838 24263 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.820886 24309 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.950629 24465 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.951128 24309 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.951285 24465 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.952509 24235 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:12.953418 24309 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.953979 24309 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:12.835916 24453 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.911834 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.954799 24453 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.958120 24453 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.942476 24435 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:12.958832 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:12.959506 16662 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:12.784098 24470 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.876024 24391 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.960603 24391 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:12.920607 16634 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.961512 16634 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:12.962723 16634 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:12.963061 24391 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:12.785519 24374 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.965063 24374 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.918187 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.908458 24519 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:21:12.966142 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.909169 24373 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.966887 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:12.967192 24373 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:12.968614 24269 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:12.969062 24269 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:12.969521 24269 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:12.921065 24508 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:12.970999 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.972792 24508 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.758976 24451 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.147s
I20260709 13:21:12.784942 24459 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:12.975206 24459 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:12.903479 24397 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:12.925366 24406 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.929014 24381 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.769992 24224 consensus_queue.cc:1048] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.141s
I20260709 13:21:12.933898 24310 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:12.949306 16667 raft_consensus.cc:1240] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:12.947171 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:12.957919 24396 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:12.960917 24164 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:12.977264 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:13.009303 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:13.009631 16738 raft_consensus.cc:2393] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 14: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:12.928876 17411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48626 (request call id 2295) took 1162 ms. Trace:
I20260709 13:21:12.784583 24279 raft_consensus.cc:697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:12.977948 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:12.988193 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:13.005393 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:12.950032 24056 raft_consensus.cc:740] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:13.010035 17411 rpcz_store.cc:276] 0709 13:21:11.766036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.768289 (+  2253us) service_pool.cc:224] Handling call
0709 13:21:11.768555 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.269615 (+501060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:12.269654 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:12.269692 (+    38us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:21:12.269727 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.928607 (+658880us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b540073b254. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:12.928663 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:12.928807 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151879936}
W20260709 13:21:13.028252 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:12.980521 24391 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:12.882334 24060 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:12.987428 24465 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:13.035885 16634 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:13.042011 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 64139.1us
W20260709 13:21:13.043421 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55302.9us
W20260709 13:21:13.044984 16634 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.045047 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:13.046851 16634 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.047930 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.055867 24432 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:13.056453 24432 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.056849 24432 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:13.059235 24525 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:13.060755 24492 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:13.062296 24525 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:13.068466 24432 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.071725 24432 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.075508 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:13.080498 24524 log.cc:927] Time spent T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.346s	user 0.002s	sys 0.001s
I20260709 13:21:13.089980 16741 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:21:13.091287 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:13.092075 16664 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:13.108887 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.113773 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:13.114931 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 60268.3us
W20260709 13:21:13.123497 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 76704us
W20260709 13:21:13.124667 16634 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.125190 16634 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.133347 16489 catalog_manager.cc:5697] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 18, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:12.989715 24519 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.098s
I20260709 13:21:13.163434 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:13.175287 16669 raft_consensus.cc:2393] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:13.167141 16634 leader_election.cc:336] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.180955 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.182907 16663 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:13.189443 16664 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
I20260709 13:21:12.990340 24397 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:12.990396 24470 consensus_queue.cc:1059] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.169s
I20260709 13:21:13.134496 24263 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:13.204315 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.211733 16667 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:13.172407 24343 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:13.242892 24517 log.cc:927] Time spent T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.507s	user 0.004s	sys 0.000s
I20260709 13:21:13.161465 24164 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.068s
I20260709 13:21:13.277626 24381 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:13.279121 24465 raft_consensus.cc:3055] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:13.217214 24432 consensus_queue.cc:1059] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:21:13.279637 24465 raft_consensus.cc:740] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:13.250172 24523 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:21:13.280836 24465 consensus_queue.cc:260] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.282238 24465 raft_consensus.cc:3060] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:13.282316 16739 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4642) took 1454 ms. Trace:
I20260709 13:21:13.282979 16739 rpcz_store.cc:276] 0709 13:21:11.827447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:11.827566 (+   119us) service_pool.cc:224] Handling call
0709 13:21:11.836686 (+  9120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:11.837048 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:11.837081 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:11.837550 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:21:12.618453 (+780903us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b18002fce00. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:21:12.731913 (+113460us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:12.731953 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:12.732031 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:13.281888 (+549857us) raft_consensus.cc:1685] finished
0709 13:21:13.281921 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:13.282239 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2060,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":14478,"spinlock_wait_cycles":9609472,"thread_start_us":113252,"threads_started":1,"wal-append.queue_time_us":1174069}
I20260709 13:21:13.214331 24279 consensus_queue.cc:237] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.312017 24056 consensus_queue.cc:260] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.313239 24056 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:13.316759 24520 raft_consensus.cc:3055] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:13.317395 24520 raft_consensus.cc:740] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:13.323263 24520 consensus_queue.cc:260] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.411056 24520 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:13.343180 24447 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:13.412261 24447 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.415503 24447 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:13.343319 24060 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:13.362656 24531 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 13:21:13.368075 16491 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 307) took 19065 ms (client timeout 15000 ms). Trace:
I20260709 13:21:13.383270 24453 raft_consensus.cc:493] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:13.423281 24453 raft_consensus.cc:515] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.340180 24073 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.424959 24073 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.428027 24073 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:13.432840 24269 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.434487 16665 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
W20260709 13:21:13.443818 16740 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.041s	sys 0.070s
W20260709 13:21:13.445124 16740 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.041s	sys 0.070s
W20260709 13:21:13.447206 17374 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.071s	sys 0.120s
W20260709 13:21:13.447552 17374 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.071s	sys 0.120s
W20260709 13:21:13.447870 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 677533us
I20260709 13:21:13.448513 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:13.448882 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 277577us
W20260709 13:21:13.451129 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 278832us
W20260709 13:21:13.451432 16709 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.453260 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 286150us
W20260709 13:21:13.454450 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:13.455204 16634 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:13.420048 16491 rpcz_store.cc:269] 0709 13:20:54.189951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:20:54.190343 (+   392us) service_pool.cc:224] Handling call
0709 13:21:13.255418 (+19065075us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:05.388275 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:05.568533 (+180258us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:05.574224 (+  5691us) write_op.cc:435] Acquired schema lock
0709 13:21:05.574238 (+    14us) tablet.cc:662] Decoding operations
0709 13:21:05.855885 (+281647us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:05.863974 (+  8089us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:05.867756 (+  3782us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:21:06.218974 (+351218us) tablet.cc:701] Row locks acquired
0709 13:21:06.225940 (+  6966us) write_op.cc:260] PREPARE: finished
0709 13:21:06.239615 (+ 13675us) write_op.cc:270] Start()
0709 13:21:06.248765 (+  9150us) write_op.cc:276] Timestamp: P: 1783603266239538 usec, L: 0
0709 13:21:06.248796 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:21:06.249555 (+   759us) log.cc:844] Serialized 2441 byte log entry
0709 13:21:07.245260 (+995705us) spinlock_profiling.cc:243] Waited 5.08 ms on lock 0x7b18001ffcc0. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:21:07.589196 (+343936us) op_driver.cc:464] REPLICATION: finished
0709 13:21:08.022487 (+433291us) write_op.cc:301] APPLY: starting
0709 13:21:08.039376 (+ 16889us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:08.221028 (+181652us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:08.221092 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:08.511101 (+290009us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:08.610377 (+ 99276us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=0,mrs_lookups=0
0709 13:21:08.633463 (+ 23086us) write_op.cc:312] APPLY: finished
0709 13:21:08.763104 (+129641us) log.cc:844] Serialized 65 byte log entry
0709 13:21:09.012607 (+249503us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:09.032107 (+ 19500us) write_op.cc:454] Released schema lock
0709 13:21:09.072713 (+ 40606us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:09.234367 (+161654us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:09.250946 (+ 16579us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":41450,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":417054,"apply.run_cpu_time_us":30543,"apply.run_wall_time_us":1170222,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29231,"num_ops":6,"prepare.queue_time_us":284751,"prepare.run_cpu_time_us":27983,"prepare.run_wall_time_us":1148002,"raft.queue_time_us":296911,"raft.run_cpu_time_us":90949,"raft.run_wall_time_us":303715,"replication_time_us":1323078,"spinlock_wait_cycles":11682176,"thread_start_us":1096032,"threads_started":5,"wal-append.queue_time_us":899417,"wal-append.run_cpu_time_us":11226,"wal-append.run_wall_time_us":2398158}]]}
I20260709 13:21:13.455688 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.472807 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.500622 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.471741 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.502051 17409 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:13.470160 17411 raft_consensus.cc:1240] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 19. Current term is 20. Ops: []
I20260709 13:21:13.487313 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:13.477135 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.476672 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:13.504259 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.477605 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.505695 17415 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:13.505859 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:13.499392 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.509589 16663 raft_consensus.cc:2393] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:13.515153 16670 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:21:13.531855 16671 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:21:13.543960 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:13.545023 16664 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:21:13.548158 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.554929 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.442299 24373 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.468572 24447 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:21:13.558413 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 496647us
I20260709 13:21:13.555641 16670 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 26.
I20260709 13:21:13.557984 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.556365 16559 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:13.564236 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.574625 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:13.517688 24051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.045s	sys 0.091s
I20260709 13:21:13.579020 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:13.590415 16738 raft_consensus.cc:2393] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:13.583246 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:13.591207 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.585311 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.595947 17412 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:21:13.509682 24406 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:13.470227 24343 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:13.586494 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.614374 17411 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 13.
I20260709 13:21:13.598733 17413 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 24. Current term is 25. Ops: []
I20260709 13:21:13.615810 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:13.558032 24306 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.003s	sys 0.014s
I20260709 13:21:13.605814 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:13.510747 24407 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:21:13.612295 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 136082us
W20260709 13:21:13.612231 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 143850us
I20260709 13:21:13.616412 16741 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:13.589779 24051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.046s	sys 0.091s
I20260709 13:21:13.619326 16742 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 26.
I20260709 13:21:13.620308 16710 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:13.570755 16743 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:21:13.618448 17419 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 16: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:13.595384 24435 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.595213 24060 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:21:13.644483 17416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.009s	sys 0.009s
W20260709 13:21:13.646209 17416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.010s	sys 0.009s
I20260709 13:21:13.614291 24451 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.615059 24447 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:21:13.654929 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 53250.4us
I20260709 13:21:13.617614 24269 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:13.618928 24306 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.004s	sys 0.014s
I20260709 13:21:13.625525 24263 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:21:13.647859 16664 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:13.621644 24164 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:13.645012 23788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.001s	sys 0.017s
I20260709 13:21:13.668382 16736 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 LEADER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 24. Current term is 25. Ops: []
W20260709 13:21:13.658156 24459 consensus_meta.cc:287] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.003s
I20260709 13:21:13.613529 24522 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:21:13.661525 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:13.677529 16559 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:13.678947 16666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.014s	sys 0.009s
W20260709 13:21:13.679365 16666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.014s	sys 0.009s
W20260709 13:21:13.680610 15476 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:13.657022 24060 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:13.625958 24406 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.639222 24407 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.682350 24366 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:21:13.663563 24360 raft_consensus.cc:2804] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:21:13.664191 24492 consensus_meta.cc:287] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.603s	user 0.012s	sys 0.000s
I20260709 13:21:13.656167 24435 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:13.667511 24320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.072s	sys 0.076s
I20260709 13:21:13.695233 16736 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37644 (request call id 4655) took 1396 ms. Trace:
W20260709 13:21:13.668502 23788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.001s	sys 0.018s
I20260709 13:21:13.662781 24396 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.027s
W20260709 13:21:13.719782 16561 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.682564 24310 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:13.662286 24451 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.714479 16736 rpcz_store.cc:276] 0709 13:21:12.298944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.301902 (+  2958us) service_pool.cc:224] Handling call
0709 13:21:12.307573 (+  5671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:13.668933 (+1361360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.694975 (+ 26042us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400049a54. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4d571 00007f327fbe4e01 00007f327fbe383e 00007f3284ba8419 00007f327e11fbd4 00007f327e11fb62 00007f327e11fa91 00007f327e11fa0c 00007f327e11ecc2 00007f327df1f48c 00007f327df1e8c6
0709 13:21:13.695171 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3128967552}
I20260709 13:21:13.669902 24164 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.693099 24366 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:13.714401 24320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.072s	sys 0.076s
I20260709 13:21:13.745637 24373 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:13.716856 24408 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.681639 24459 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.726333 24470 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.727455 24453 leader_election.cc:290] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.726625 24360 raft_consensus.cc:697] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:13.730849 24397 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.741274 24366 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:13.740404 24310 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:13.770038 17414 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 24: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:13.747123 24452 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:13.768127 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 260264us
I20260709 13:21:13.747594 24306 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:13.783166 24306 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.785604 24306 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:13.767724 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 254446us
I20260709 13:21:13.780120 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:13.787829 17374 leader_election.cc:336] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.788388 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.791157 24263 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:13.791354 24269 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.792073 24263 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:13.792161 24269 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.793596 24263 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.829530 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.780613 24406 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.797457 24269 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.732589 24447 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.795038 24391 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:13.804332 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.782147 24164 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:13.820833 24459 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:13.822770 24435 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.824111 24360 consensus_queue.cc:237] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.805348 17416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48636 (request call id 1247) took 1273 ms. Trace:
I20260709 13:21:13.801728 24451 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:13.853950 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 66163.3us
I20260709 13:21:13.858831 24224 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:13.859226 24534 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:13.791205 24507 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 13:21:13.886034 24534 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.759943 24408 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.857260 24373 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:13.873826 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 418637us
I20260709 13:21:13.863473 24320 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:13.878953 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.889174 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:13.890399 16634 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.891003 16634 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.883832 16741 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 17.
I20260709 13:21:13.885083 24224 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.886318 24507 consensus_queue.cc:1243] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer efa00eb5e7404570958fdebb6cd077e8 log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 13:21:13.765749 24470 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.768919 24060 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.874819 17416 rpcz_store.cc:276] 0709 13:21:12.532022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.532180 (+   158us) service_pool.cc:224] Handling call
0709 13:21:12.532514 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:12.532885 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:12.532916 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:12.533399 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:21:13.093729 (+560330us) spinlock_profiling.cc:243] Waited 16.3 ms on lock 0x7b1800450160. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cad9e07 00007f3285934ab9 00007f327caf669b 00007f327caf493f 00007f327caf6c02 00007f327f77e20b 00007f327f783d23 00007f327f7840d1 00007f327fb9fa0f
0709 13:21:13.652741 (+559012us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:13.652806 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.652915 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:13.804926 (+152011us) raft_consensus.cc:1685] finished
0709 13:21:13.804955 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:13.805260 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":721,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5004,"spinlock_wait_cycles":37412224,"thread_start_us":558697,"threads_started":1,"wal-append.queue_time_us":1251626}
W20260709 13:21:13.891657 17374 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.899111 17374 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:13.899272 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 254432us
W20260709 13:21:13.900738 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.901278 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:13.845701 24452 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:13.904551 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 259469us
I20260709 13:21:13.906427 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:13.910679 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.870882 24391 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.915175 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:13.916050 17419 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
W20260709 13:21:13.932118 24536 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:21:13.934090 24535 log.cc:927] Time spent T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:21:13.935281 24543 log.cc:927] Time spent T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:21:13.939534 24530 log.cc:927] Time spent T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:21:13.942222 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:13.942047 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.943153 24164 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:13.943678 24164 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.944028 24164 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:13.944293 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:13.945585 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:13.946671 24548 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:13.947636 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:13.947870 24279 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:13.948474 24279 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:13.948961 24279 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:13.947664 24548 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:13.943215 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:13.954983 24306 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:13.955274 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.957038 24320 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:13.957876 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:13.957813 24320 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.960330 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:13.888264 24373 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.963860 24474 consensus_queue.cc:1059] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:13.888368 24310 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:13.966470 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.968286 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:13.969837 24306 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:13.970335 24306 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:13.971359 24056 consensus_meta.cc:287] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.000s
I20260709 13:21:13.906060 24532 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:13.903709 24507 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:13.973313 24507 raft_consensus.cc:3055] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:13.973691 24507 raft_consensus.cc:740] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:13.974491 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:13.974583 24507 consensus_queue.cc:260] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:13.977349 24470 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:13.977494 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:13.978376 16736 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:21:13.963901 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:13.987082 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:13.988322 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:13.911535 16634 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:13.895211 24451 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:13.993580 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 82099.5us
I20260709 13:21:14.027448 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.005601 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.046350 17419 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:13.999224 17416 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
W20260709 13:21:14.040814 16634 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:14.048775 16634 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.050215 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.004536 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:13.908077 24459 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:14.051275 17414 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:13.997961 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.057255 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.004765 24056 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:14.026213 24534 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.078344 17414 raft_consensus.cc:2468] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 14.
I20260709 13:21:14.080340 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.019951 24546 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 13:21:14.146193 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.147567 17413 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:14.133239 24459 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:14.166743 24407 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.161204 24459 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.175347 24507 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:14.175568 24459 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:13.996596 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:14.213099 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:14.217398 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 220733us
W20260709 13:21:14.217847 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 219847us
I20260709 13:21:14.219424 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:14.222584 16561 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.231256 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:14.235842 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 231859us
I20260709 13:21:14.234839 16561 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:14.238061 16710 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:14.241503 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.245448 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:14.248502 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.154975 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:14.257604 16740 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37636 (request call id 2968) took 1341 ms. Trace:
I20260709 13:21:14.260593 17411 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:21:14.262924 17410 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:14.258598 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:14.265810 17413 raft_consensus.cc:1240] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: []
I20260709 13:21:14.254459 16738 raft_consensus.cc:1275] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:21:14.287174 23788 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:14.293974 24056 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:14.297318 24073 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.298903 24269 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:21:14.300190 24310 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:21:14.301242 24347 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:14.301615 24343 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:14.343562 24432 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:14.346951 24397 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:14.349920 24396 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:14.350826 24396 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:14.353322 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 105153us
I20260709 13:21:14.353097 24396 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:14.355167 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 109946us
I20260709 13:21:14.346988 24073 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.355351 24520 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:21:14.361069 24520 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.361620 24520 raft_consensus.cc:3060] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:21:14.165318 24408 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.365680 24552 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:14.356696 24051 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:14.367300 24553 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:14.367623 24051 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.368299 24051 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:14.368155 24553 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.369031 24391 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.303323 16740 rpcz_store.cc:276] 0709 13:21:12.915840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:12.916122 (+   282us) service_pool.cc:224] Handling call
0709 13:21:12.926127 (+ 10005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:21:12.926347 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:21:13.446434 (+520087us) log.cc:844] Serialized 14 byte log entry
0709 13:21:13.755916 (+309482us) log.cc:844] Serialized 14 byte log entry
0709 13:21:13.886147 (+130231us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b18002ba020. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327cac52a6 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1
0709 13:21:13.887614 (+  1467us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b18002ba020. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:21:13.968243 (+ 80629us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18002ba020. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327ca71c0a 00007f32861b1219 00007f327caf9a40 00007f327caf44e4 00007f327caf4455 00007f32835dc3c7 00007f327fbee9a7 00007f327fbfe0bf 00007f327fbfe038
0709 13:21:13.968580 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:21:14.256129 (+287549us) log.cc:844] Serialized 14 byte log entry
0709 13:21:14.256536 (+   407us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:21:14.256642 (+   106us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:21:14.256783 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:14.256951 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:14.257289 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157941,"prepare.queue_time_us":639972,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":1620,"raft.queue_time_us":209410,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":9943,"spinlock_wait_cycles":39870464,"thread_start_us":1051610,"threads_started":3,"wal-append.queue_time_us":465170}
I20260709 13:21:14.374128 24553 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:14.384516 24532 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:14.358517 24453 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:14.389573 24452 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.389890 17419 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 17: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:14.367547 24552 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:21:14.390411 24320 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.392989 17409 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 24: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:14.399477 24073 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.402040 24373 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.390887 24532 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.405997 24532 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:14.412127 24541 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:21:14.412815 24541 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:14.413558 24452 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:14.415321 24347 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:14.392033 24552 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:14.417409 24552 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.414574 24541 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.433573 24258 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:21:14.441615 24538 log.cc:927] Time spent T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.545s	user 0.002s	sys 0.001s
I20260709 13:21:14.446305 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:14.446909 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:14.447238 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.447860 16665 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 24: Already voted for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in this term.
I20260709 13:21:14.453835 24552 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:21:14.456658 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 99158.8us
I20260709 13:21:14.434715 24258 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.457856 16710 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:14.485045 24492 raft_consensus.cc:2804] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:21:14.485929 24492 raft_consensus.cc:697] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:14.490659 16666 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36082 (request call id 3975) took 1415 ms. Trace:
I20260709 13:21:14.491199 16666 rpcz_store.cc:276] 0709 13:21:13.074789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:13.075031 (+   242us) service_pool.cc:224] Handling call
0709 13:21:13.075518 (+   487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:21:13.076008 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:21:13.076050 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:21:13.076791 (+   741us) log.cc:844] Serialized 26 byte log entry
0709 13:21:13.679730 (+602939us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:21:13.679770 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:21:13.679839 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:21:14.440764 (+760925us) spinlock_profiling.cc:243] Waited 49.7 ms on lock 0x7b240010dd38. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327c97d9c4 00007f327c97bacb 00007f327cb36dc9 00007f327cb36a55 00007f327cb3609d 00007f327f76810e 00007f327f77f88c 00007f327f77ebb6 00007f327f77e7d8 00007f327f788221
0709 13:21:14.490054 (+ 49290us) raft_consensus.cc:1685] finished
0709 13:21:14.490078 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:21:14.490429 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1427,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":114420608,"thread_start_us":602495,"threads_started":1,"wal-append.queue_time_us":787808}
I20260709 13:21:14.488049 24492 consensus_queue.cc:237] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:14.508745 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 467806us
I20260709 13:21:14.509943 24397 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:14.510610 16634 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 132 similar messages]
I20260709 13:21:14.511018 24397 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.513979 24224 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.515769 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:14.515241 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 102664us
I20260709 13:21:14.501741 24541 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:14.534452 24279 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.585s	user 0.000s	sys 0.019s
W20260709 13:21:14.508821 24366 consensus_meta.cc:287] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.683s	user 0.018s	sys 0.007s
I20260709 13:21:14.535841 16710 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:14.537771 24306 consensus_meta.cc:287] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.008s
I20260709 13:21:14.538987 24073 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:14.541381 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.552248 24269 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:14.514808 24397 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:14.584129 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 73577.4us
W20260709 13:21:14.601963 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 66261.7us
I20260709 13:21:14.606115 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.587438 24366 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.585541 24306 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.640434 24347 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.660445 24347 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.661731 24306 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.663028 16742 raft_consensus.cc:2393] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:14.614923 17376 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:14.664803 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 167711us
I20260709 13:21:14.665733 17376 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.667214 24559 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:14.668078 24559 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:21:14.582494 24279 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.671190 24279 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.672199 16663 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:14.672734 24258 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.652096 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.652856 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.655683 24269 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:14.644414 24308 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:14.647814 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:14.647814 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 106599us
I20260709 13:21:14.645859 24572 consensus_queue.cc:1048] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:21:14.636488 24397 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:14.679510 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:14.679116 24269 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.698105 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:14.695482 17376 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:14.699571 24224 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:14.700073 24224 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.680537 16739 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
W20260709 13:21:14.681046 24164 consensus_meta.cc:287] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.014s
I20260709 13:21:14.701624 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.701807 24164 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.702423 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.704061 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.704447 24164 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.704960 16741 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20. Ops: [17.14-19.15]
I20260709 13:21:14.705457 24343 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:14.682200 24347 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:14.683174 15476 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:14.683776 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.705891 24520 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.614570 24051 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.707592 16667 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 15. Current term is 16. Ops: []
I20260709 13:21:14.708912 24051 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.709172 17376 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.700469 24224 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:14.713001 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.691571 24366 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.724821 24073 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.725553 24073 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.727978 24073 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.713872 16666 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:14.731930 24520 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.735165 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:14.735526 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:14.737389 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.696049 24269 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:14.766975 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:14.769976 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 60952.4us
I20260709 13:21:14.771893 24373 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:14.773259 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.775146 17376 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.775614 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:14.776326 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.778569 24534 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:14.779330 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:14.783239 16708 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.694s (SENT)
I20260709 13:21:14.784144 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:14.794888 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:14.796936 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.797463 24320 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:14.798379 24320 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:21:14.798727 24320 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:14.787122 24347 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:21:14.799767 24320 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.801187 24320 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:14.802388 24073 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.792384 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.803160 24073 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.804292 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:14.806242 24073 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.807476 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:14.820063 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.824389 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:14.831486 17413 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:14.832402 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.839020 16668 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:21:14.836484 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:14.845482 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.846503 16670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:14.847224 16670 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:14.850069 24060 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.850852 24060 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.854689 24060 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:14.855508 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.856328 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.857972 16667 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:21:14.861354 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:14.863240 23788 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:14.863636 23788 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:14.863950 23788 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.859951 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
W20260709 13:21:14.875746 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49744us
I20260709 13:21:14.839787 24343 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:14.886579 16671 raft_consensus.cc:1240] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
W20260709 13:21:14.889101 16708 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.784s (SENT)
I20260709 13:21:14.893546 24073 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.894058 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:14.895051 16667 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 13:21:14.894290 24073 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.897951 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:14.895413 16667 pending_rounds.cc:85] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 17
I20260709 13:21:14.883335 24492 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:14.904819 16741 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:14.900650 16667 pending_rounds.cc:107] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 13:21:14.909482 16667 raft_consensus.cc:2889] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:14.904772 24572 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:21:14.916682 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 56805.6us
I20260709 13:21:14.921804 16739 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:14.894997 24453 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
W20260709 13:21:14.931717 17374 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:14.932258 17374 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.936311 24373 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:14.925379 24451 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:14.909453 24492 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.944406 16670 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 27.
I20260709 13:21:14.952095 24520 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:14.952886 24520 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:14.991074 24373 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:14.991868 24373 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:14.994316 24373 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:14.997268 24343 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:14.997910 24343 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:14.999330 24572 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:14.999867 24572 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:15.007532 23788 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.011930 16739 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 20. Current term is 21. Ops: []
W20260709 13:21:15.013971 24549 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.101s	user 0.002s	sys 0.002s
I20260709 13:21:15.015528 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:14.955291 24451 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.020033 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be6c4adb5a2a4a34b6c89e15242b20a1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.003450 24073 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.022475 16559 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:15.033524 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 129320us
W20260709 13:21:15.037386 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 76559.4us
I20260709 13:21:15.011965 24567 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:15.022294 24451 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:15.033751 24532 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:15.047783 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.048439 23788 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.041141 24073 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:21:15.057338 17413 raft_consensus.cc:2468] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 23.
W20260709 13:21:15.061710 17374 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.059500 16708 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:15.068328 24366 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.069005 24366 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.067853 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:15.084937 24567 consensus_queue.cc:1059] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:21:15.087714 24532 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:15.088017 24567 raft_consensus.cc:3055] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:15.088421 24567 raft_consensus.cc:740] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:15.071223 24520 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:15.089325 24567 consensus_queue.cc:260] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.067024 24073 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
W20260709 13:21:15.089723 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 207409us
W20260709 13:21:15.095467 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.100572 17419 raft_consensus.cc:2410] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:21:15.097096 16736 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:15.102546 16742 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.080662 24552 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:15.097085 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:15.094990 24595 log.cc:927] Time spent T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:21:15.121951 16710 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:15.090837 24567 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:15.125612 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:15.124502 24488 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
I20260709 13:21:15.133896 17414 raft_consensus.cc:1275] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:21:15.134224 16743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.130398 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.138211 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:15.148903 16708 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.971s (SENT)
I20260709 13:21:15.149457 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:15.130275 24306 raft_consensus.cc:2804] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:15.154122 24306 raft_consensus.cc:697] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 23 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:15.134104 24548 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:15.155105 24306 consensus_queue.cc:237] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.155814 24548 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.158085 24548 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.159677 24492 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:15.092038 24224 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.140866 24366 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:15.164062 24592 log.cc:927] Time spent T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:21:15.165472 24463 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:21:15.167891 16634 leader_election.cc:336] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.168655 16634 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:15.143788 24534 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:15.171622 24343 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.176252 16665 raft_consensus.cc:1240] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:15.178421 24343 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:15.179041 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 137247us
I20260709 13:21:15.181002 16736 raft_consensus.cc:2410] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 13:21:15.191716 17410 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
W20260709 13:21:15.192095 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 53994.5us
I20260709 13:21:15.100455 24572 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.198444 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:15.201792 16671 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:15.204720 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:15.180984 24224 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:15.206357 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.091771 24073 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.194604 24548 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:15.222878 16736 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:15.216847 16662 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:15.225437 16669 raft_consensus.cc:1240] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:15.227833 24507 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:15.229254 17411 raft_consensus.cc:1275] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:21:15.230337 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.230551 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:15.231237 16740 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:15.232690 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.200721 24164 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:15.244029 16739 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:21:15.243983 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:15.251130 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 83260.7us
W20260709 13:21:15.251922 16634 leader_election.cc:336] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.252461 16634 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:15.219587 24452 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.245675 24453 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:21:15.266970 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 86446.1us
I20260709 13:21:15.235387 24534 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:21:15.278187 16740 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 27.
W20260709 13:21:15.281487 17374 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Timed out: RequestConsensusVote RPC to 127.15.29.1:34889 timed out after 1.603s (SENT)
I20260709 13:21:15.282255 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:15.288902 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:15.290149 24594 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.283s	user 0.005s	sys 0.000s
W20260709 13:21:15.295068 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 146186us
W20260709 13:21:15.296530 16708 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.297556 24343 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:21:15.297988 16708 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.300081 24343 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:15.300486 24343 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:15.302695 24343 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.304059 24343 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:15.312583 24366 consensus_queue.cc:1059] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:21:15.313614 24366 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:15.314877 24396 raft_consensus.cc:3055] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:15.315485 24396 raft_consensus.cc:740] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:15.316565 24396 consensus_queue.cc:260] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.317857 24396 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:15.319229 24366 consensus_queue.cc:1059] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:15.318957 24453 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.321909 24453 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.323009 24366 raft_consensus.cc:3055] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:15.323592 24366 raft_consensus.cc:740] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:15.324965 24366 consensus_queue.cc:260] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.326417 24366 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:15.280040 24060 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.327105 24374 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:15.327283 24060 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.342060 16671 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:15.267094 24269 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:15.347939 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:15.334462 16736 raft_consensus.cc:2468] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 24.
I20260709 13:21:15.328047 24374 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.372107 23788 consensus_queue.cc:1048] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:21:15.379551 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:15.359979 24224 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:21:15.383448 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 85531.1us
I20260709 13:21:15.349601 24534 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:21:15.386170 23788 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:15.369647 24567 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:21:15.332002 24452 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.345145 24164 consensus_queue.cc:1048] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:21:15.371239 24453 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:15.403182 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 121756us
I20260709 13:21:15.415279 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.387269 24224 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:15.418879 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:15.414033 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.425170 24606 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.429939 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.441028 23788 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:15.442058 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.440189 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:15.419353 24572 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:15.445793 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 57289.5us
I20260709 13:21:15.447132 17376 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:15.456063 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.467047 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.475219 24347 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.476017 24347 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.476296 16487 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 12 to 13, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 13 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:15.478760 24459 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
W20260709 13:21:15.479485 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 86144.2us
I20260709 13:21:15.469383 24492 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:15.485951 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 235) took 7877 ms. Trace:
W20260709 13:21:15.488276 24602 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:15.490077 23788 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.492682 24060 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:15.479059 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.490495 24374 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.494783 16487 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 18 to 19, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 19 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.441365 24453 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.493816 24373 raft_consensus.cc:2804] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:15.540966 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.478775 24347 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:15.545569 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 111994us
W20260709 13:21:15.544207 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44693.3us
W20260709 13:21:15.547832 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 97268.9us
I20260709 13:21:15.553485 17409 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:15.464898 24606 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.565975 16561 leader_election.cc:304] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:15.567525 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 139978us
I20260709 13:21:15.567682 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.567128 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.575163 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.575917 17410 raft_consensus.cc:2393] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in current term 23: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:15.557010 24373 raft_consensus.cc:697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 24 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:15.583477 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.590623 16487 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 16 to 18, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:15.592664 16487 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:15.595535 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.596323 17412 raft_consensus.cc:2468] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
W20260709 13:21:15.574329 24609 log.cc:927] Time spent T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:21:15.618811 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.625051 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:15.648799 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:15.651296 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 85422.6us
I20260709 13:21:15.486495 16489 rpcz_store.cc:276] 0709 13:21:07.619270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:07.631723 (+ 12453us) service_pool.cc:224] Handling call
0709 13:21:08.298542 (+666819us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b5400014854. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327fb4e090 00007f327fbecf3a 00007f32859a31cc 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:21:08.402829 (+104287us) spinlock_profiling.cc:243] Waited 51.7 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cac0640 00007f3285997b76 00007f3285aa1c3e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc 00007f3283f61392 00007f327df1f48c 00007f327df1e8c6
0709 13:21:15.485688 (+7082859us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:13.859247 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:13.859838 (+   591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:13.859880 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:21:13.859900 (+    20us) tablet.cc:662] Decoding operations
0709 13:21:13.861070 (+  1170us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:13.861140 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:13.861193 (+    53us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:21:13.862052 (+   859us) tablet.cc:701] Row locks acquired
0709 13:21:13.862076 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:21:13.862268 (+   192us) write_op.cc:270] Start()
0709 13:21:13.862397 (+   129us) write_op.cc:276] Timestamp: P: 1783603273862209 usec, L: 0
0709 13:21:13.862426 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:21:13.913468 (+ 51042us) log.cc:844] Serialized 3170 byte log entry
0709 13:21:14.333577 (+420109us) op_driver.cc:464] REPLICATION: finished
0709 13:21:14.542232 (+208655us) write_op.cc:301] APPLY: starting
0709 13:21:14.542371 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:14.681869 (+139498us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:14.682035 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:14.941861 (+259826us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:14.947541 (+  5680us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:21:14.947613 (+    72us) write_op.cc:312] APPLY: finished
0709 13:21:14.949206 (+  1593us) log.cc:844] Serialized 81 byte log entry
0709 13:21:14.949452 (+   246us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:14.951850 (+  2398us) write_op.cc:454] Released schema lock
0709 13:21:14.951918 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:14.952558 (+   640us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:14.952601 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2936041,"reported_tablets":15,"spinlock_wait_cycles":624600704,"child_traces":[["op",{"apply.queue_time_us":208208,"apply.run_cpu_time_us":18772,"apply.run_wall_time_us":410329,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37599,"num_ops":8,"prepare.queue_time_us":655613,"prepare.run_cpu_time_us":31155,"prepare.run_wall_time_us":306487,"raft.queue_time_us":153491,"raft.run_cpu_time_us":19653,"raft.run_wall_time_us":139518,"replication_time_us":470830,"thread_start_us":938913,"threads_started":4,"wal-append.queue_time_us":193465}]]}
I20260709 13:21:15.649544 17419 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
I20260709 13:21:15.621760 24608 raft_consensus.cc:2749] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:15.706388 15476 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:15.709226 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.721570 24606 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.702962 24452 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:15.725054 24617 log.cc:927] Time spent T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 13:21:15.725831 24612 log.cc:927] Time spent T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:21:15.683386 24279 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.735873 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:15.739020 17374 leader_election.cc:336] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.738744 24373 consensus_queue.cc:237] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:15.740381 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:15.741372 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:15.744196 16560 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.744632 24279 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.748980 24224 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.749363 16487 catalog_manager.cc:5697] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 13 to 15, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 15 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:15.750166 24224 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:15.753057 17419 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 13:21:15.753463 17419 pending_rounds.cc:85] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 17
I20260709 13:21:15.753690 17419 pending_rounds.cc:107] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 13:21:15.753907 17419 raft_consensus.cc:2889] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:15.759043 24453 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.759486 24614 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.760023 24614 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:15.760807 24626 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:21:15.762216 24614 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:15.763057 24606 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.763595 24606 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.765636 24606 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:15.768647 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.769990 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:15.774309 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:15.775213 24452 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:15.775516 24627 log.cc:927] Time spent T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.074s	user 0.000s	sys 0.002s
I20260709 13:21:15.775764 24279 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.776491 24452 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:15.778376 24366 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:21:15.781311 24619 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.781872 24619 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:15.784281 24619 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:15.785725 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.787050 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:15.779986 16561 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:15.788137 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46981us
W20260709 13:21:15.794107 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 105172us
I20260709 13:21:15.795053 23788 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:15.795619 23788 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:15.791260 16561 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:15.796941 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 207731us
I20260709 13:21:15.798420 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:15.788638 24552 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:15.800690 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.793380 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.808180 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.824643 24366 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:15.813225 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:15.803174 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:15.831003 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.718101 24397 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 13:21:15.796569 24606 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:15.832633 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:15.827323 16665 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:15.823060 16669 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:15.856976 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.851877 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.862654 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.833134 23788 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.844332 24347 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:15.882154 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.890677 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:15.898313 24603 log.cc:927] Time spent T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:21:15.901522 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.907413 17410 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 24 LEADER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:21:15.821820 24552 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:15.911207 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:15.908620 17374 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:15.917435 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:15.912952 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:15.923673 16669 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:21:15.924093 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:15.922647 16737 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:15.924814 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:15.933152 24619 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:15.941457 24619 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.941871 24619 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:15.944641 24552 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:15.948791 24396 consensus_meta.cc:287] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.001s
I20260709 13:21:15.929421 24606 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:15.949723 24606 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:15.950102 24606 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:15.956593 24616 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:15.965042 24452 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:15.972986 24343 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:15.973680 24343 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:15.973706 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56282.3us
I20260709 13:21:15.975865 24343 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:15.988059 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:15.989163 16671 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:21:15.994745 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:15.995599 16737 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
W20260709 13:21:16.001206 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 151016us
I20260709 13:21:16.004253 17410 raft_consensus.cc:1275] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:21:15.984284 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.010154 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:16.029978 16708 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:16.025478 24343 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:21:16.046159 16634 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.047232 24051 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:16.049804 24343 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
W20260709 13:21:16.067361 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:16.072090 24366 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:16.072271 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 88345us
W20260709 13:21:16.073050 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:16.072739 24366 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.073642 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.074337 17420 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:21:16.036906 24488 consensus_queue.cc:1048] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:21:16.077232 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:16.071171 16634 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.075032 24619 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:16.064144 24051 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.075875 24343 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.100777 24051 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:21:16.145092 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 62592.1us
W20260709 13:21:16.186149 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.188414 16671 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:21:16.189903 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:16.190467 16671 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.190995 16739 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:21:16.191577 24616 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:16.192338 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 125595us
I20260709 13:21:16.192142 24616 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.194195 24616 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:16.196283 16634 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.201455 16668 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:21:16.201840 16668 pending_rounds.cc:85] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 13
I20260709 13:21:16.202047 16668 pending_rounds.cc:107] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:21:16.202265 16668 raft_consensus.cc:2889] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:16.204169 24629 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.204725 24629 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:16.207576 24629 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.213137 17413 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:16.213829 24051 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.215737 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.223652 17413 raft_consensus.cc:2393] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 24: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:16.217046 24606 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.218596 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:16.217543 24619 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.219977 24616 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.220530 24620 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:16.221616 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.224673 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.232923 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.234922 16671 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
I20260709 13:21:16.240396 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.240592 16559 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:16.241967 24347 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.251725 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.251879 24347 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:16.254688 24347 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.257674 16744 raft_consensus.cc:1275] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:21:16.258288 16708 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.260249 16634 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.269096 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.270605 24459 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:16.296482 24606 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.271243 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ecf0f1df23742a2a06e910e6b7350a0" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.296716 24459 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.276086 16736 raft_consensus.cc:1240] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Rejecting Update request from peer 3c3ca2a0390b46bb9a5eda03a64eda1a for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:21:16.264526 24396 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:16.299178 24396 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:16.300266 24396 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:16.301209 24459 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.302304 24051 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.266666 24366 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.284650 17413 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:21:16.307646 16736 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:21:16.265890 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:16.264576 17419 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
W20260709 13:21:16.306727 16480 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 894a0fdf94344f46b0426efbd37a6c40 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER failed with leader ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica ba2df6a668144022a223e504ae571f21 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:16.310365 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 65698.8us
I20260709 13:21:16.311699 24620 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:16.312685 24620 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.313338 24620 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.199810 24343 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.319224 16664 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:16.323499 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:16.323734 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:16.324359 16664 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:16.324707 16666 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:16.240630 24616 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.334007 24616 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.336053 24343 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:16.341208 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 51080.1us
W20260709 13:21:16.351276 16634 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.359297 16634 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.363634 16634 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.368381 16666 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
W20260709 13:21:16.370131 16562 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:21:16.371827 16559 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:16.377331 16664 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:16.388062 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:16.372852 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:16.391288 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:16.391820 16739 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.388229 24073 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:21:16.401890 16708 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.360824 24606 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.360489 24629 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:16.406287 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 134586us
I20260709 13:21:16.407399 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:16.374493 24507 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:21:16.410843 16708 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.414651 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:16.415493 17419 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:16.390341 24279 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:16.421747 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:16.405516 24619 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.422719 16744 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:16.425906 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:16.426462 17416 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:16.430071 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:16.431265 16736 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:16.429520 24451 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:16.424336 24629 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.417511 24459 consensus_queue.cc:1048] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:21:16.480187 24629 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.447417 24624 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.481529 24624 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.484833 24624 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.445369 24343 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.495730 24451 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.496412 24451 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.428568 24279 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.501400 24279 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.504027 24629 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.504894 24629 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:16.511421 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 104199us
W20260709 13:21:16.511515 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 120289us
W20260709 13:21:16.464247 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 53414.3us
I20260709 13:21:16.470418 17419 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
W20260709 13:21:16.518204 16708 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.519573 24620 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.436650 24619 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:16.419890 24606 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.513895 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:16.461078 24366 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:16.522564 16708 leader_election.cc:336] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.536410 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:16.536737 24654 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:16.537498 23788 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:21:16.538779 16708 leader_election.cc:336] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.539392 16708 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:16.540297 23788 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:16.541146 16708 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.523639 16561 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:16.541879 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 137572us
I20260709 13:21:16.525839 16739 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
I20260709 13:21:16.535738 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:21:16.543103 16634 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.544255 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.545197 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.545960 16664 raft_consensus.cc:2393] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:16.552692 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:16.555053 16708 leader_election.cc:336] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.555490 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:16.553297 16665 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 19.
I20260709 13:21:16.556120 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:16.552158 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:16.558133 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.543156 24619 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.563953 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.549674 24572 raft_consensus.cc:2764] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:21:16.566288 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.568224 24606 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.558813 24366 raft_consensus.cc:3060] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:16.575613 23788 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:16.551083 24073 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:16.565330 24373 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 13:21:16.585610 24373 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:21:16.585084 24451 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.586632 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:16.591488 17419 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21.
I20260709 13:21:16.591236 17376 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:16.593017 16667 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.599524 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:16.605352 24606 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:16.605537 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:16.599836 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:16.611534 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 56528.1us
W20260709 13:21:16.612347 16708 leader_election.cc:336] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.586428 24629 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.612772 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:16.591063 24373 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:16.617058 24452 raft_consensus.cc:2804] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:16.623168 24452 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:16.623668 24452 raft_consensus.cc:3060] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.614748 24646 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:21:16.600032 24306 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:21:16.603052 24616 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:16.628407 24616 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:16.628913 24616 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:16.639030 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:16.641917 16561 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.622143 24373 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.644042 24606 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:21:16.645169 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.645988 24629 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.647277 16561 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:16.648084 24629 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:16.648483 24629 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.648820 24629 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:16.658535 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.661651 16636 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21.
I20260709 13:21:16.661729 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:16.665652 24373 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.667438 17413 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:21:16.667850 17413 pending_rounds.cc:85] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Aborting all ops after (but not including) 13
I20260709 13:21:16.668085 17413 pending_rounds.cc:107] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:21:16.668308 17413 raft_consensus.cc:2889] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:16.670107 24646 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:16.671887 16744 raft_consensus.cc:2393] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 20: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:16.662053 16636 leader_election.cc:403] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:16.674993 24646 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.677479 24463 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 21
W20260709 13:21:16.678148 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63973.1us
I20260709 13:21:16.680349 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:16.695256 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 82954.1us
W20260709 13:21:16.696604 16708 leader_election.cc:336] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:16.697175 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:16.702015 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 43563.2us
I20260709 13:21:16.703238 23788 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:16.705262 16708 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.707245 15476 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 16621)
W20260709 13:21:16.733747 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52046us
W20260709 13:21:16.710906 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:16.736120 16708 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:16.739603 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:16.694669 24606 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:16.745653 16666 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:21:16.745688 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.748028 16665 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:16.738096 24552 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:16.736833 24051 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
W20260709 13:21:16.768003 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 84201.2us
W20260709 13:21:16.771049 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 59235us
I20260709 13:21:16.772596 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.759701 24308 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:16.776072 16561 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.779367 24606 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:16.783504 24606 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:21:16.775031 24343 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:16.737825 24073 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.795846 24366 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:16.801553 16561 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:16.802911 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 55580.8us
I20260709 13:21:16.784839 24656 consensus_queue.cc:1059] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
I20260709 13:21:16.777722 24653 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:21:16.813521 16561 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.813211 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.787972 24308 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:16.815649 24343 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:16.816928 24308 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:16.818553 24452 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.806177 24451 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:16.821053 24452 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.822662 24656 consensus_queue.cc:1243] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Peer ba2df6a668144022a223e504ae571f21 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:21:16.823241 24452 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:16.823325 24656 raft_consensus.cc:3055] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:21:16.823848 24656 raft_consensus.cc:740] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:16.824903 24656 consensus_queue.cc:260] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.826262 24656 raft_consensus.cc:3060] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:16.836292 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:16.837044 16741 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:16.842519 24492 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:21:16.845278 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:16.845894 24657 log.cc:927] Time spent T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:21:16.846535 24051 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:16.847033 24051 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:16.847414 24051 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:16.807806 24396 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:21:16.861248 24629 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.863770 24629 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.866387 24051 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:16.861207 16741 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:16.877597 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.887663 16663 raft_consensus.cc:2393] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 23: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:16.878919 17376 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:16.894109 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 80653.4us
I20260709 13:21:16.895305 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.923758 24051 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:16.924546 24656 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:16.926513 16561 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:16.927875 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 49072.7us
I20260709 13:21:16.924448 16492 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 18 to 20, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:16.957171 24308 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:16.958006 24308 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:16.960062 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 77806.8us
I20260709 13:21:16.964146 16561 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:16.970530 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 87787.8us
I20260709 13:21:16.963238 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:16.972927 16663 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:16.968358 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 181055us
I20260709 13:21:16.967114 24224 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:16.963238 24656 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:16.977101 24653 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:16.992316 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:16.994294 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:16.996016 16671 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:16.991513 16561 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:16.981778 24629 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.030170 16663 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:17.009779 24463 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 20. Current term is 21.
W20260709 13:21:17.001278 16487 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 314) took 13842 ms (client timeout 15000 ms). Trace:
I20260709 13:21:17.020380 24308 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:17.025233 24397 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:17.031963 16671 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 25.
I20260709 13:21:16.997454 24224 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:17.048461 16665 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:16.995577 16492 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 13 to 16, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 16 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:17.053705 16708 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:17.050418 24659 log.cc:927] Time spent T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:21:16.987609 24655 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 13:21:17.052363 24656 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:17.039739 16487 rpcz_store.cc:269] 0709 13:21:03.152597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:03.153005 (+   408us) service_pool.cc:224] Handling call
0709 13:21:16.994635 (+13841630us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:15.893688 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:15.894511 (+   823us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:15.894572 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:21:15.894594 (+    22us) tablet.cc:662] Decoding operations
0709 13:21:15.904826 (+ 10232us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:15.904923 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:15.904958 (+    35us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:21:15.905732 (+   774us) tablet.cc:701] Row locks acquired
0709 13:21:15.905780 (+    48us) write_op.cc:260] PREPARE: finished
0709 13:21:15.905989 (+   209us) write_op.cc:270] Start()
0709 13:21:15.906182 (+   193us) write_op.cc:276] Timestamp: P: 1783603275905908 usec, L: 0
0709 13:21:15.906205 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:21:15.917201 (+ 10996us) log.cc:844] Serialized 2059 byte log entry
0709 13:21:16.222512 (+305311us) op_driver.cc:464] REPLICATION: finished
0709 13:21:16.499158 (+276646us) write_op.cc:301] APPLY: starting
0709 13:21:16.499252 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:16.507379 (+  8127us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:16.507537 (+   158us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:16.516204 (+  8667us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:16.550145 (+ 33941us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:21:16.553444 (+  3299us) write_op.cc:312] APPLY: finished
0709 13:21:16.575012 (+ 21568us) log.cc:844] Serialized 57 byte log entry
0709 13:21:16.577825 (+  2813us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.577936 (+   111us) write_op.cc:454] Released schema lock
0709 13:21:16.577996 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:16.578501 (+   505us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:16.578535 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":117231,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":275388,"apply.run_cpu_time_us":10400,"apply.run_wall_time_us":79302,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":110685,"prepare.run_cpu_time_us":4723,"prepare.run_wall_time_us":35778,"replication_time_us":297350,"thread_start_us":248918,"threads_started":2}]]}
I20260709 13:21:17.073081 16559 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:17.077044 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:17.070499 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 79057.3us
I20260709 13:21:17.069803 24308 raft_consensus.cc:2804] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:17.091079 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.094310 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.096415 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.101392 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.102062 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.102432 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.103574 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c74caaa7fe045f9a0f6ff3b3df97602" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.104308 16736 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:21:17.105547 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.102018 16492 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 19 to 20, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.109251 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:17.109840 17416 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:21:17.112365 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.114559 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.114953 16492 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 18 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:17.125744 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.084250 24656 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.130017 16492 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a reported cstate change: term changed from 17 to 20, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:17.139580 17374 leader_election.cc:336] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.110463 24308 raft_consensus.cc:697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:21:17.130807 24616 consensus_meta.cc:287] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.000s
W20260709 13:21:17.156477 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.157123 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.156802 24308 consensus_queue.cc:237] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.134744 24224 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.133656 24656 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:17.166428 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 93415.4us
I20260709 13:21:17.131278 24672 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:17.186108 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.189252 16671 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 21.
I20260709 13:21:17.208964 17416 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:17.210595 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
W20260709 13:21:17.214352 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57910.2us
W20260709 13:21:17.215169 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.215591 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.155194 24616 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.188558 24656 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.228175 24656 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:17.230240 24656 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:17.190037 24397 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.230955 24397 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.232285 24629 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:17.232707 24629 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.233038 24629 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:17.233697 24397 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:17.192790 24662 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:21:17.223446 24667 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:21:17.208016 24672 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:17.238205 24672 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.210021 24451 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:17.244686 24671 log.cc:927] Time spent T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:21:17.246477 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 176952us
I20260709 13:21:17.259173 16668 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:17.271039 24662 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.287053 16666 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:17.247510 17415 raft_consensus.cc:2468] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
W20260709 13:21:17.305685 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 44110.2us
I20260709 13:21:17.310600 17411 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:21:17.312265 23788 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:21:17.317317 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 102178us
W20260709 13:21:17.318082 17374 leader_election.cc:336] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.318502 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.323748 24616 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:17.327299 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:17.327104 24647 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:21:17.341022 17374 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.341722 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.339792 24451 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:17.348879 24616 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:17.353298 17374 leader_election.cc:336] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.353910 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:17.355257 17374 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.355410 16708 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.355695 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.355942 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.356477 24662 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:17.356947 24164 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:17.357486 16708 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.357915 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:17.358094 17374 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.358549 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.359869 24051 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:17.363049 24662 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:17.364238 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.364758 24678 log.cc:927] Time spent T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:21:17.356514 24616 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:17.372777 16561 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:17.373761 24656 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.375396 16561 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.376217 24656 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:17.383279 24656 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:17.384073 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.384756 16744 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:17.385634 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.385571 16561 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.364676 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.391774 24656 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:17.392037 24616 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:17.393415 17374 leader_election.cc:336] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.394930 16561 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:17.400326 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 189798us
I20260709 13:21:17.401820 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.402449 16668 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 23.
I20260709 13:21:17.404980 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.404901 24655 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:17.423419 24655 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:17.423839 24655 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:17.433789 24629 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.436026 24629 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:17.454658 16671 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:21:17.456291 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50763.9us
W20260709 13:21:17.466571 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 70355.7us
I20260709 13:21:17.476600 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:17.477404 16668 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:17.473771 16559 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:17.480113 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.481845 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:17.483676 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:17.492055 24680 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:17.481119 24656 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:17.497756 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 103415us
W20260709 13:21:17.499648 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:17.495482 24308 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:17.503893 24655 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:17.516862 16668 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:17.519843 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:17.520076 16665 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:21:17.520634 16671 raft_consensus.cc:2368] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20.
I20260709 13:21:17.521493 16559 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:17.530334 24451 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:17.532516 24451 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.533022 24396 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:17.530133 24224 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:17.533756 24396 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:17.533823 24224 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.534886 24396 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.536095 24224 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:17.536129 24655 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:17.541671 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:17.548398 16561 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:17.554243 24662 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:17.557060 16663 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:17.560055 23788 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:17.573074 16561 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.575078 17374 leader_election.cc:400] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20.
I20260709 13:21:17.576303 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:17.575698 17374 leader_election.cc:403] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:21:17.577421 16668 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:17.580631 24663 raft_consensus.cc:3060] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:17.555002 24662 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.583436 24680 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:17.584733 24662 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:17.586617 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.592466 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8b0d179ad714b59b34ab1dc57a57c02" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:17.592979 24680 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:17.570464 24672 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 13:21:17.575697 24683 log.cc:927] Time spent T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:21:17.607124 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:17.607872 17416 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:17.609709 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.610467 17419 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 20.
I20260709 13:21:17.611804 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.613035 16561 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:17.615217 24680 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:17.616470 16561 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.617611 17419 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:21:17.620244 24451 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:17.621646 24663 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 19. Current term is 20.
I20260709 13:21:17.621521 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:17.622593 17419 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:21:17.630594 24680 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:17.631408 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.632143 16671 raft_consensus.cc:2368] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 22.
I20260709 13:21:17.632936 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.647912 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:17.650139 17374 leader_election.cc:400] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 22.
I20260709 13:21:17.650532 17374 leader_election.cc:403] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:17.634694 17419 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 22.
I20260709 13:21:17.655709 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:17.661577 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:17.671540 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.672752 16561 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:17.675730 24396 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:21:17.679399 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:17.687139 24662 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639) with higher term. Message: Invalid argument: T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 22.
I20260709 13:21:17.687099 24680 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:17.690851 24663 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:17.691823 24663 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:17.693135 17419 raft_consensus.cc:1275] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:17.694597 23788 consensus_queue.cc:1048] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:17.699349 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:17.699911 16741 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:21:17.709679 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:17.710350 15476 thread.cc:527] Waited for 9000ms trying to join with heartbeat (tid 16621)
I20260709 13:21:17.710279 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.711112 24616 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:17.716533 16489 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 18 to 19, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:17.719461 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:17.719136 16489 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 19 to 20, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.736326 16489 catalog_manager.cc:5697] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.785022 16666 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:21:17.785476 17416 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 25.
I20260709 13:21:17.739248 16489 catalog_manager.cc:5697] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 10 to 15, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.777225 16668 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
I20260709 13:21:17.793534 16741 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c3ca2a0390b46bb9a5eda03a64eda1a in term 24.
W20260709 13:21:17.802558 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:17.791447 16489 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.810555 16559 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 
I20260709 13:21:17.796452 24396 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 13:21:17.819016 24655 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:17.819522 24655 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:17.821663 24655 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:17.822187 24672 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:17.822842 24672 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 LEADER]: Becoming Leader. State: Replica: 3c3ca2a0390b46bb9a5eda03a64eda1a, State: Running, Role: LEADER
I20260709 13:21:17.826851 24672 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:17.806095 24647 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:17.842957 24647 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.846050 24647 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:17.846899 16492 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38206 (request call id 337) took 12820 ms (client timeout 15000 ms). Trace:
I20260709 13:21:17.848063 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:17.848758 17413 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:17.849663 16489 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 23 to 25, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 25 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:17.850795 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:17.854001 16489 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 20, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.852586 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:17.855744 24258 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:17.856122 24258 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:17.856487 24258 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:17.865556 16489 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 14 to 17, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.868678 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.869824 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:17.870396 16489 catalog_manager.cc:5697] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.872700 16662 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:17.806907 24699 log.cc:927] Time spent T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:21:17.874640 16489 catalog_manager.cc:5697] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 19 to 22, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 22 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:17.880496 16708 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.882838 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:17.884593 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:17.867724 24258 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:17.900077 24620 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:17.901758 24258 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:17.904749 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:17.908789 16708 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:17.909273 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:17.910041 24164 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:17.914482 16741 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:17.921373 17411 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:17.920217 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:17.918192 16489 catalog_manager.cc:5697] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 19 to 20, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:17.847332 16492 rpcz_store.cc:269] 0709 13:21:05.086806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:05.092951 (+  6145us) service_pool.cc:224] Handling call
0709 13:21:17.846795 (+12753844us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:17.415096 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:17.415869 (+   773us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:17.415920 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:21:17.415945 (+    25us) tablet.cc:662] Decoding operations
0709 13:21:17.416824 (+   879us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:17.416944 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:17.416985 (+    41us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:21:17.417685 (+   700us) tablet.cc:701] Row locks acquired
0709 13:21:17.417712 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:21:17.417925 (+   213us) write_op.cc:270] Start()
0709 13:21:17.418066 (+   141us) write_op.cc:276] Timestamp: P: 1783603277417842 usec, L: 0
0709 13:21:17.418094 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:21:17.435162 (+ 17068us) log.cc:844] Serialized 2059 byte log entry
0709 13:21:17.485631 (+ 50469us) op_driver.cc:464] REPLICATION: finished
0709 13:21:17.578507 (+ 92876us) write_op.cc:301] APPLY: starting
0709 13:21:17.578593 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:17.638833 (+ 60240us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:17.638975 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:17.642019 (+  3044us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:17.642221 (+   202us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:21:17.642259 (+    38us) write_op.cc:312] APPLY: finished
0709 13:21:17.642471 (+   212us) log.cc:844] Serialized 57 byte log entry
0709 13:21:17.642634 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:17.642747 (+   113us) write_op.cc:454] Released schema lock
0709 13:21:17.642788 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:17.669895 (+ 27107us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:17.669928 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3599,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":92598,"apply.run_cpu_time_us":7974,"apply.run_wall_time_us":91370,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25047,"num_ops":5,"prepare.queue_time_us":245166,"prepare.run_cpu_time_us":4364,"prepare.run_wall_time_us":26414,"replication_time_us":66267,"thread_start_us":178351,"threads_started":2}]]}
I20260709 13:21:17.923763 16662 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 21: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:17.925377 16710 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:17.929531 16489 catalog_manager.cc:5697] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.952868 16668 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:21:17.954702 16489 catalog_manager.cc:5697] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 21 to 23, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 23 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:17.961946 16489 catalog_manager.cc:5697] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 11 to 14, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 14 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.934217 24374 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:17.971072 16489 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 19, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 19 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.929643 24616 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:21:17.983003 24451 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:17.978212 16489 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.991874 16489 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 19 to 21, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 21 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:17.998327 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:17.994990 16489 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:18.006887 16489 catalog_manager.cc:5697] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 20 to 23, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 23 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:17.994516 24374 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
W20260709 13:21:18.022805 16559 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 86926.5us
I20260709 13:21:18.030159 16666 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
W20260709 13:21:18.038611 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 104894us
I20260709 13:21:18.029919 24704 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.051836 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.048177 24704 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:18.062201 24704 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:18.063949 17415 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 13:21:18.066057 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:18.067015 17374 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:18.067214 24374 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:18.067922 17374 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:18.072970 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:18.075760 17413 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:21:18.104190 24396 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:21:18.132966 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:18.153419 16561 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 51667.1us
I20260709 13:21:18.167049 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:18.199935 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.202317 15476 tablet_replica.cc:333] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.204510 15476 raft_consensus.cc:2243] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.186412 24704 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:21:18.217645 24712 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.032s
I20260709 13:21:18.243991 16561 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.252224 16561 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.261082 15476 pending_rounds.cc:70] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trying to abort 1 pending ops.
I20260709 13:21:18.262490 15476 pending_rounds.cc:77] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 19 index: 13 } timestamp: 7305638961297096704 op_type: NO_OP noop_request { }
I20260709 13:21:18.262943 15476 raft_consensus.cc:2889] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:18.263337 15476 raft_consensus.cc:2272] T e91194e070834298ba7ca9d0bdacc3d7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.264274 24396 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:18.268085 15476 tablet_replica.cc:333] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.268473 24703 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:18.270784 15476 raft_consensus.cc:2243] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.271510 15476 raft_consensus.cc:2272] T 42839b9fe9ae47a5a548348ea9b4eee1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.274451 15476 tablet_replica.cc:333] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.308197 15476 raft_consensus.cc:2243] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.308853 15476 raft_consensus.cc:2272] T b40781eb2d304f9b86f5061ae572e1c7 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:18.308952 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:18.312469 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.335862 15476 tablet_replica.cc:333] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.337394 15476 raft_consensus.cc:2243] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.338101 15476 raft_consensus.cc:2272] T d104b6d16beb411c9099f23901cebbb2 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:18.327307 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.346755 15476 tablet_replica.cc:333] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.352730 16744 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 13:21:18.348291 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.355054 24716 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:21:18.353752 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:18.363063 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.382064 24655 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.382606 24655 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.384676 24655 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.386324 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.386974 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.388793 16561 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.389011 15476 raft_consensus.cc:2243] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.389757 24655 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:18.391250 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.402807 15476 raft_consensus.cc:2272] T be6c4adb5a2a4a34b6c89e15242b20a1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.406152 15476 tablet_replica.cc:333] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.409876 15476 raft_consensus.cc:2243] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.410720 15476 raft_consensus.cc:2272] T 3551e95f0ec5435f84de7b8f75069b13 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.429160 15476 tablet_replica.cc:333] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.430583 15476 raft_consensus.cc:2243] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.431183 16671 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Refusing update from remote peer 3c3ca2a0390b46bb9a5eda03a64eda1a: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:21:18.436470 15476 raft_consensus.cc:2272] T f919f8e7484c4f359aedc6ac05b45076 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.437696 24655 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:18.440742 15476 tablet_replica.cc:333] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.442068 15476 raft_consensus.cc:2243] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.455309 15476 raft_consensus.cc:2272] T 3326ab1787714e708abfea6cf0509109 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.458590 15476 tablet_replica.cc:333] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.459905 15476 raft_consensus.cc:2243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.460631 15476 raft_consensus.cc:2272] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.463778 15476 tablet_replica.cc:333] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.464860 15476 raft_consensus.cc:2243] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.465534 15476 raft_consensus.cc:2272] T 98936dee7edb4aa5873923a47699a89c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.466506 24716 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.467092 24716 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.469440 24716 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.471503 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.470894 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.470955 15476 tablet_replica.cc:333] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.475157 15476 raft_consensus.cc:2243] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.475788 15476 raft_consensus.cc:2272] T e8b0d179ad714b59b34ab1dc57a57c02 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.478353 16561 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.478915 15476 tablet_replica.cc:333] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
W20260709 13:21:18.464951 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.479854 15476 raft_consensus.cc:2243] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.481220 15476 raft_consensus.cc:2272] T 894a0fdf94344f46b0426efbd37a6c40 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.492550 15476 tablet_replica.cc:333] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.493569 15476 raft_consensus.cc:2243] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.494256 15476 raft_consensus.cc:2272] T 3249701d37834e80bbe408e24ced7d54 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.494947 24655 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:18.497705 15476 tablet_replica.cc:333] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.498780 15476 raft_consensus.cc:2243] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.499706 15476 raft_consensus.cc:2272] T 9fe018a9ac6d466983da014c7b86d36f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:18.509590 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:18.513593 15476 tablet_replica.cc:333] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.514487 24716 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.514786 15476 raft_consensus.cc:2243] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.515287 24716 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.516434 15476 raft_consensus.cc:2272] T 73089175ee9c465197a1a11e304346be P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.518939 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.519610 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.519884 15476 tablet_replica.cc:333] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.518363 24716 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.521896 16561 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:18.520880 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:18.520892 15476 raft_consensus.cc:2243] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.531771 24655 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:18.532572 24655 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.533066 24655 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.533840 24710 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.535285 24655 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.536087 15476 raft_consensus.cc:2272] T 51e96ec0851c4732b9a3e1272e9fd80f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.539287 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.534523 24710 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.545229 24710 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.539340 15476 tablet_replica.cc:333] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.547768 15476 raft_consensus.cc:2243] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.548707 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.549170 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.550974 16561 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.551832 16671 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:21:18.552175 24655 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:18.549336 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.553207 17413 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:18.554104 24374 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:18.554592 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:18.555750 24664 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:18.556166 24664 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:18.556537 24664 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:18.548743 15476 raft_consensus.cc:2272] T 0ecf0f1df23742a2a06e910e6b7350a0 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.561189 24716 raft_consensus.cc:493] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:18.563354 24716 raft_consensus.cc:515] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.566593 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.568082 15476 tablet_replica.cc:333] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.569139 15476 raft_consensus.cc:2243] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.597009 24716 leader_election.cc:290] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.599525 15476 raft_consensus.cc:2272] T 10731e2d513f466faf0d3f2030af5469 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.604110 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc977d4c44af4e80b58e4e0864085d9a" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.605033 15476 tablet_replica.cc:333] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.611991 16561 leader_election.cc:304] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.615818 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 236) took 1803 ms. Trace:
I20260709 13:21:18.616652 16489 rpcz_store.cc:276] 0709 13:21:16.847264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:16.847508 (+   244us) service_pool.cc:224] Handling call
0709 13:21:18.615731 (+1768223us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:18.121925 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:18.122472 (+   547us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:18.122514 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:21:18.122532 (+    18us) tablet.cc:662] Decoding operations
0709 13:21:18.125309 (+  2777us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:18.125373 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:18.125406 (+    33us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:21:18.127128 (+  1722us) tablet.cc:701] Row locks acquired
0709 13:21:18.127145 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:21:18.127301 (+   156us) write_op.cc:270] Start()
0709 13:21:18.127412 (+   111us) write_op.cc:276] Timestamp: P: 1783603278127253 usec, L: 0
0709 13:21:18.127433 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:21:18.128076 (+   643us) log.cc:844] Serialized 7500 byte log entry
0709 13:21:18.210546 (+ 82470us) op_driver.cc:464] REPLICATION: finished
0709 13:21:18.253126 (+ 42580us) write_op.cc:301] APPLY: starting
0709 13:21:18.253217 (+    91us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:18.399623 (+146406us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:18.399749 (+   126us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:18.471142 (+ 71393us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:18.471439 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 13:21:18.471485 (+    46us) write_op.cc:312] APPLY: finished
0709 13:21:18.471876 (+   391us) log.cc:844] Serialized 172 byte log entry
0709 13:21:18.472035 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:18.472216 (+   181us) write_op.cc:454] Released schema lock
0709 13:21:18.472267 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:18.472697 (+   430us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:18.472724 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":83672,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":42290,"apply.run_cpu_time_us":28045,"apply.run_wall_time_us":219524,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83623,"num_ops":19,"prepare.queue_time_us":61856,"prepare.run_cpu_time_us":5010,"prepare.run_wall_time_us":16259,"replication_time_us":82915,"thread_start_us":43985,"threads_started":2}]]}
I20260709 13:21:18.619527 17413 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:21:18.627255 16491 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 18 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:18.629317 16491 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 22 to 24, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 24 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:18.630769 16491 catalog_manager.cc:5697] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 20 to 21, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 21 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:18.638257 24715 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.638900 24715 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:18.641148 24715 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:18.632205 16491 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:18.642814 24620 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:18.645890 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:18.642633 16491 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 20 to 21, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 21 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:18.647863 16491 catalog_manager.cc:5697] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 18 to 19, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 19 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:18.652493 16491 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 14 to 15, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:18.654115 16491 catalog_manager.cc:5697] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:18.659952 15476 raft_consensus.cc:2243] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.663363 16491 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 26 to 27, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 27 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:18.665494 16491 catalog_manager.cc:5697] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 21 to 23, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 23 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:18.653345 24716 raft_consensus.cc:2749] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:18.669595 24664 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.671662 16671 raft_consensus.cc:1240] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:21:18.671901 24664 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.686093 16491 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 13 to 15, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:18.692853 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:18.699321 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:18.690687 24647 consensus_queue.cc:1059] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 13:21:18.709486 24647 raft_consensus.cc:3055] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:18.709954 24647 raft_consensus.cc:740] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:18.711125 24647 consensus_queue.cc:260] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.712587 24647 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:18.717841 15476 pending_rounds.cc:70] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trying to abort 1 pending ops.
I20260709 13:21:18.718080 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:18.722134 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:18.718256 15476 pending_rounds.cc:77] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305639005475450880 op_type: NO_OP noop_request { }
I20260709 13:21:18.723227 15476 raft_consensus.cc:2889] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:18.723798 15476 raft_consensus.cc:2272] T 173ca11ec288406aa68fe01a80d48e30 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.734169 16561 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:18.747493 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.737860 24648 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:18.752604 24716 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:18.792747 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52421.5us
I20260709 13:21:18.787495 24712 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:18.804503 24712 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:18.826604 24712 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:18.827950 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:18.857970 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.861537 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:18.866179 16561 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:18.866652 15476 tablet_replica.cc:333] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.867507 24712 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:18.867723 15476 raft_consensus.cc:2243] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:18.868453 15476 pending_rounds.cc:70] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trying to abort 2 pending ops.
I20260709 13:21:18.868709 15476 pending_rounds.cc:77] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 14 index: 12 } timestamp: 7305638981460758528 op_type: NO_OP noop_request { }
I20260709 13:21:18.869073 15476 raft_consensus.cc:2889] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:18.869385 15476 pending_rounds.cc:77] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 12 index: 11 } timestamp: 7305638953124274176 op_type: NO_OP noop_request { }
I20260709 13:21:18.869721 15476 raft_consensus.cc:2889] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 13:21:18.869810 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:18.870112 15476 raft_consensus.cc:2272] T ecb93faa15cf4f7b853b030dfdc6af28 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:18.888314 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:18.913182 15476 tablet_replica.cc:333] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.918260 15476 raft_consensus.cc:2243] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.929497 15476 raft_consensus.cc:2272] T e5e5fb284dac45b2a4c12e2ec38a1784 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.949788 15476 tablet_replica.cc:333] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:18.979980 15476 raft_consensus.cc:2243] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:21:18.983312 15476 raft_consensus.cc:2272] T 40930ae2df144d709bfb017b86dfbf2b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:18.990386 15476 tablet_replica.cc:333] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.004108 16487 catalog_manager.cc:5697] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:19.024405 16487 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:19.039134 16491 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 308) took 3936 ms. Trace:
I20260709 13:21:19.039673 16487 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:19.039558 16491 rpcz_store.cc:276] 0709 13:21:15.103197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:15.103529 (+   332us) service_pool.cc:224] Handling call
0709 13:21:15.162908 (+ 59379us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5400002490. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f327c97fdb1 00007f327cabfb71 00007f327cac00b2 00007f3285a39a89 00007f32859d1da6 00007f32859a3088 00007f3285aa137e 00007f3283f622a4 00007f3283f62232 00007f3283f62161 00007f3283f620dc
0709 13:21:19.039038 (+3876130us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:18.836798 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:18.837336 (+   538us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:18.837372 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:21:18.837389 (+    17us) tablet.cc:662] Decoding operations
0709 13:21:18.838369 (+   980us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:18.838452 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:18.838478 (+    26us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:21:18.844270 (+  5792us) tablet.cc:701] Row locks acquired
0709 13:21:18.844289 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:21:18.844430 (+   141us) write_op.cc:270] Start()
0709 13:21:18.844544 (+   114us) write_op.cc:276] Timestamp: P: 1783603278844390 usec, L: 0
0709 13:21:18.844566 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:21:18.845195 (+   629us) log.cc:844] Serialized 4346 byte log entry
0709 13:21:18.885164 (+ 39969us) op_driver.cc:464] REPLICATION: finished
0709 13:21:18.885558 (+   394us) write_op.cc:301] APPLY: starting
0709 13:21:18.885641 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:18.893328 (+  7687us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:18.893429 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:18.932327 (+ 38898us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:18.932560 (+   233us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 13:21:18.932606 (+    46us) write_op.cc:312] APPLY: finished
0709 13:21:18.932907 (+   301us) log.cc:844] Serialized 105 byte log entry
0709 13:21:18.933098 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:18.933232 (+   134us) write_op.cc:454] Released schema lock
0709 13:21:18.933287 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:18.933671 (+   384us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:18.933697 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":67919,"reported_tablets":44,"spinlock_wait_cycles":2973440,"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":15465,"apply.run_wall_time_us":48130,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46024,"num_ops":11,"prepare.queue_time_us":131162,"prepare.run_cpu_time_us":3922,"prepare.run_wall_time_us":15404,"replication_time_us":40417,"thread_start_us":130836,"threads_started":1}]]}
I20260709 13:21:19.044369 15476 raft_consensus.cc:2243] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.045039 15476 raft_consensus.cc:2272] T 5bfdfe1aa047409c9214ba1f43eb434a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.058048 16487 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:19.059958 16487 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 21 to 24, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 24 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:19.064284 16487 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:19.065804 16487 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:19.067571 16487 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:19.068701 15476 tablet_replica.cc:333] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.073158 15476 raft_consensus.cc:2243] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.074152 15476 raft_consensus.cc:2272] T bab84fe76da941dc98416ce3f8c6f35b P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:19.107029 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.116511 15476 tablet_replica.cc:333] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.117552 15476 raft_consensus.cc:2243] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.118187 15476 pending_rounds.cc:70] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trying to abort 1 pending ops.
I20260709 13:21:19.118475 15476 pending_rounds.cc:77] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 20 index: 17 } timestamp: 7305638961801515008 op_type: NO_OP noop_request { }
I20260709 13:21:19.118836 15476 raft_consensus.cc:2889] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:19.135396 15476 raft_consensus.cc:2272] T d02ec9cc775248dfa8321ae141d0ad37 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.135883 16489 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:19.138572 15476 tablet_replica.cc:333] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.138041 16489 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:19.155978 16489 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 19 to 20, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 20 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:19.160418 16489 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:19.161931 16489 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:19.155622 15476 raft_consensus.cc:2243] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.163172 16489 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 22 to 24, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1). New cstate: current_term: 24 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:19.164072 15476 raft_consensus.cc:2272] T 446649301a1c45808dccfebfadfed54c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.150838 16487 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38212 (request call id 315) took 1316 ms. Trace:
I20260709 13:21:19.167632 16487 rpcz_store.cc:276] 0709 13:21:17.834251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:17.834418 (+   167us) service_pool.cc:224] Handling call
0709 13:21:19.150769 (+1316351us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:19.080339 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:19.080864 (+   525us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:19.080900 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:21:19.080919 (+    19us) tablet.cc:662] Decoding operations
0709 13:21:19.081831 (+   912us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:19.081883 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:19.081920 (+    37us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:21:19.082781 (+   861us) tablet.cc:701] Row locks acquired
0709 13:21:19.082802 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:21:19.082989 (+   187us) write_op.cc:270] Start()
0709 13:21:19.083122 (+   133us) write_op.cc:276] Timestamp: P: 1783603279082944 usec, L: 0
0709 13:21:19.083146 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:21:19.083842 (+   696us) log.cc:844] Serialized 3186 byte log entry
0709 13:21:19.086112 (+  2270us) op_driver.cc:464] REPLICATION: finished
0709 13:21:19.086394 (+   282us) write_op.cc:301] APPLY: starting
0709 13:21:19.086474 (+    80us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:19.110963 (+ 24489us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:19.111061 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:19.117979 (+  6918us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:19.118233 (+   254us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:21:19.118277 (+    44us) write_op.cc:312] APPLY: finished
0709 13:21:19.118611 (+   334us) log.cc:844] Serialized 81 byte log entry
0709 13:21:19.119429 (+   818us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:19.119546 (+   117us) write_op.cc:454] Released schema lock
0709 13:21:19.119597 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:19.119972 (+   375us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:19.120008 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10345,"reported_tablets":38,"spinlock_wait_cycles":21376,"child_traces":[["op",{"apply.queue_time_us":97,"apply.run_cpu_time_us":10759,"apply.run_wall_time_us":33538,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33472,"num_ops":8,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":3733,"prepare.run_wall_time_us":3816,"replication_time_us":2822}]]}
I20260709 13:21:19.175033 15476 tablet_replica.cc:333] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.176076 15476 raft_consensus.cc:2243] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.176684 15476 raft_consensus.cc:2272] T d9aa91533ea94ae392057ef582e0c419 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.203938 15476 tablet_replica.cc:333] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.205030 15476 raft_consensus.cc:2243] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.205696 15476 raft_consensus.cc:2272] T ff94f30d9bba44ee9059993ecbbbf398 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:19.218617 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.222182 15476 tablet_replica.cc:333] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.223241 15476 raft_consensus.cc:2243] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.223862 15476 raft_consensus.cc:2272] T cc977d4c44af4e80b58e4e0864085d9a P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.226711 15476 tablet_replica.cc:333] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.227573 15476 raft_consensus.cc:2243] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.228109 15476 pending_rounds.cc:70] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trying to abort 1 pending ops.
I20260709 13:21:19.228344 15476 pending_rounds.cc:77] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 19 index: 15 } timestamp: 7305638938052493312 op_type: NO_OP noop_request { }
I20260709 13:21:19.228691 15476 raft_consensus.cc:2889] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:19.229030 15476 raft_consensus.cc:2272] T fc0ed8d0a60640a0a5ce3a772c3eef5c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.248713 24712 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:19.251173 16491 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } }
I20260709 13:21:19.249256 24712 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:19.261440 24712 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:19.262128 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:19.264240 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:19.267184 16561 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c3ca2a0390b46bb9a5eda03a64eda1a; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:19.271127 24712 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:19.282917 15476 tablet_replica.cc:333] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.283921 15476 raft_consensus.cc:2243] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.284497 15476 pending_rounds.cc:70] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Trying to abort 1 pending ops.
I20260709 13:21:19.284742 15476 pending_rounds.cc:77] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a: Aborting op as it isn't in flight: id { term: 22 index: 15 } timestamp: 7305638995262607360 op_type: NO_OP noop_request { }
I20260709 13:21:19.287417 15476 raft_consensus.cc:2889] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:21:19.287793 15476 raft_consensus.cc:2272] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.305977 15476 tablet_replica.cc:333] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.307119 15476 raft_consensus.cc:2243] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.307926 16489 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 20 to 21, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 21 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:19.308348 15476 raft_consensus.cc:2272] T f1a71b94ecfb4155a5cfe591077e9a1c P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.348449 15476 tablet_replica.cc:333] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.351228 15476 raft_consensus.cc:2243] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.354830 15476 raft_consensus.cc:2272] T 56335fb09b2846e184d61194e1d17f74 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.380026 15476 tablet_replica.cc:333] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.380961 15476 raft_consensus.cc:2243] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.382403 15476 raft_consensus.cc:2272] T ee99829f455541f09dac21015b00282e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.385898 15476 tablet_replica.cc:333] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.442505 15476 raft_consensus.cc:2243] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.443744 15476 raft_consensus.cc:2272] T 92d5dbc15e7c4691b7ef572175476c3d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.452582 15476 tablet_replica.cc:333] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.453377 15476 raft_consensus.cc:2243] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.488327 15476 raft_consensus.cc:2272] T 9b74fd3415d14ae3aeb38c349eca0494 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.491158 15476 tablet_replica.cc:333] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
W20260709 13:21:19.510629 17374 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 213 similar messages]
I20260709 13:21:19.492026 15476 raft_consensus.cc:2243] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.512204 15476 raft_consensus.cc:2272] T 92129e39cb38498ea9f0addc0a33f247 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.524394 15476 tablet_replica.cc:333] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.525261 15476 raft_consensus.cc:2243] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.525974 15476 raft_consensus.cc:2272] T 9f202d4fdd58454a9c8e006ea35dbc46 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.535596 15476 tablet_replica.cc:333] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.540558 15476 raft_consensus.cc:2243] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.541460 15476 raft_consensus.cc:2272] T 6d92a8b6dd5a4c64b749d3505ca2033e P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.545080 15476 tablet_replica.cc:333] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.548017 15476 raft_consensus.cc:2243] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.548604 15476 raft_consensus.cc:2272] T 8243711a48fd4d8c85d79e7c10801c4f P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.553359 15476 tablet_replica.cc:333] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.555003 15476 raft_consensus.cc:2243] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.555802 15476 raft_consensus.cc:2272] T 8b778f817264440987b35333e6540c72 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.575021 24616 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.575683 24616 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.576622 15476 tablet_replica.cc:333] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.577895 15476 raft_consensus.cc:2243] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.578567 15476 raft_consensus.cc:2272] T 2125e4b55933488bbb6e25dba4189adb P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.579937 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:19.580747 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:19.585544 15476 tablet_replica.cc:333] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.602988 24616 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:19.603823 15476 raft_consensus.cc:2243] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.605191 15476 raft_consensus.cc:2272] T 0445daac2da44dc5af84baa8514bb195 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.629907 15476 tablet_replica.cc:333] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.630743 15476 raft_consensus.cc:2243] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:19.639756 15476 raft_consensus.cc:2272] T 79e87765622246cc907a538b35c52c64 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.659566 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:19.660420 24731 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:19.665979 15476 tablet_replica.cc:333] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.678570 15476 raft_consensus.cc:2243] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:19.679785 15476 raft_consensus.cc:2272] T bb360c38974b4352a4febd1e0aada45d P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:19.704288 15476 tablet_replica.cc:333] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a: stopping tablet replica
I20260709 13:21:19.705088 15476 raft_consensus.cc:2243] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:21:19.705912 17374 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.715718 15476 raft_consensus.cc:2272] T f836a80f84054393b6251b24eaf890f1 P 3c3ca2a0390b46bb9a5eda03a64eda1a [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:19.718910 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:19.720425 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:19.734041 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:19.786296 24726 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.786964 24726 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.788805 24731 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.789414 24731 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.790472 24647 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.791097 24647 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:19.791548 24731 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:19.793677 24647 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:19.794764 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:19.798137 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:19.799446 24728 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.799997 24728 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:19.802385 16708 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.802711 24728 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:19.804126 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:19.806308 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.806777 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:19.808439 24730 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:19.810310 24710 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.811198 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:19.811954 17415 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:19.811228 24710 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.819087 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:19.819840 17415 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:19.822094 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:19.823175 24648 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:19.823257 24710 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:19.823748 24648 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.824245 24648 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:19.829373 16708 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.829883 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:19.832029 24728 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:19.832931 16634 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:19.834714 16708 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.836870 16708 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:19.840548 24728 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:19.849071 24726 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:19.850351 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:19.852113 16710 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:19.853909 24726 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:19.856186 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:19.869707 24710 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.870402 24710 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.872032 24734 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.872718 24734 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:19.874511 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:19.875378 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:19.876528 24734 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:19.877981 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:19.878749 16741 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
I20260709 13:21:19.880110 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:19.882848 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:19.885023 24734 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:19.885475 24734 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.885553 24710 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:19.885883 24734 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:21:19.892257 16634 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.892841 16634 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:19.899153 24731 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.899824 24731 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.902781 24731 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:19.903584 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:19.882192 24648 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.904453 16671 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:19.905467 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:19.905972 24648 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:19.906061 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:19.909451 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:19.909524 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:19.910676 17413 raft_consensus.cc:3060] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:19.911635 24710 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:19.912811 17374 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.913168 24710 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.913754 24731 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:19.913676 24710 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.914186 24731 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.914635 24731 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:19.917542 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:19.918203 16741 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:19.922161 24710 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:19.928977 24728 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.929659 24728 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:19.956290 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 64079us
I20260709 13:21:19.957028 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:19.957137 16634 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.959339 17415 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:19.958640 17413 raft_consensus.cc:2468] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:19.960688 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:19.964071 24731 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:19.965880 16708 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:19.970221 17374 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:19.973956 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:19.974810 16741 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:19.979815 24648 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:19.980274 16636 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:19.980405 24648 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:19.981297 24710 raft_consensus.cc:2804] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:19.981981 24710 raft_consensus.cc:697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:19.994972 24728 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:19.995496 24731 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:19.982937 24710 consensus_queue.cc:237] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:19.994114 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.001973 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.003551 16662 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 24.
I20260709 13:21:20.001394 16671 raft_consensus.cc:3060] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:20.005029 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.007524 24647 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:20.007936 24647 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.008440 24647 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:20.009749 24734 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.014995 16741 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:19.999645 24648 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:21:20.028175 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:20.037729 24734 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.042368 24647 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:20.047137 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:20.050033 16671 raft_consensus.cc:2468] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
W20260709 13:21:20.053818 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58685.6us
W20260709 13:21:20.054714 16634 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:20.055640 16634 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.059955 17374 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.063409 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:20.064064 16744 raft_consensus.cc:3060] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:20.067813 24731 raft_consensus.cc:2804] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:20.068439 24731 raft_consensus.cc:697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:20.069362 24731 consensus_queue.cc:237] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.088279 24647 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.091115 16744 raft_consensus.cc:2468] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 25.
I20260709 13:21:20.095042 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.095785 16671 raft_consensus.cc:3060] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:20.104285 16489 catalog_manager.cc:5697] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 22 to 23, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 23 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.112967 16671 raft_consensus.cc:2468] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 25.
W20260709 13:21:20.118741 16708 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:20.120255 16634 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.124246 16636 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.127521 16708 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.128959 24733 raft_consensus.cc:2804] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:20.129532 24733 raft_consensus.cc:697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 25 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:20.130378 24733 consensus_queue.cc:237] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.133132 24734 raft_consensus.cc:2804] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:21:20.136122 24734 raft_consensus.cc:697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 25 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:21:20.136665 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:20.132519 24736 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.143777 24736 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.146026 24736 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.153221 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:20.162097 24732 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:20.162750 24732 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:20.167066 16634 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.167723 24732 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.137039 24734 consensus_queue.cc:237] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.175228 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:20.180405 24737 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:20.187986 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.188781 16671 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:20.182055 16487 catalog_manager.cc:5697] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 24 to 25, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 25 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.194716 24710 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.195547 24710 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.173837 24733 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.251245 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:20.252038 16744 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:20.253424 24710 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.261751 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:20.262732 24730 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:20.263273 24730 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:20.263746 24730 raft_consensus.cc:3060] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:20.272746 24730 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.275971 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.276289 24730 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.276793 16671 raft_consensus.cc:2393] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:20.281333 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:20.231460 24733 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.285917 24733 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.292356 24648 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.293396 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:20.295150 16708 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.301678 16671 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 27.
I20260709 13:21:20.245304 16491 catalog_manager.cc:5697] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 21 to 22, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 22 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.303138 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.308192 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:20.310130 17413 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:20.312896 24648 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.317217 16744 raft_consensus.cc:2468] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:20.320186 17376 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:20.318089 24648 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.324998 24648 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.327590 17413 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:20.327710 24648 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.329329 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:20.324378 24731 raft_consensus.cc:2804] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:20.330905 24731 raft_consensus.cc:697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:21:20.331976 16634 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.331796 24731 consensus_queue.cc:237] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:20.333096 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:20.334023 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:20.318037 24733 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 13:21:20.338335 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:20.315881 24647 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:21:20.341732 16634 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.342486 24647 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.346076 24647 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.356069 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:20.356746 16741 raft_consensus.cc:3060] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:20.357231 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.357566 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:20.357812 16671 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:20.358278 16744 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:21:20.350196 16489 catalog_manager.cc:5697] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 24 to 25, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 25 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:20.356691 16634 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.361744 16634 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:20.339468 24733 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.374578 24733 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:21:20.380409 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:20.382891 16741 raft_consensus.cc:2468] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 20.
I20260709 13:21:20.391714 24749 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:20.392899 24730 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.393461 24730 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.395747 24730 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.397255 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:20.397897 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:20.394605 24748 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:20.399849 24748 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.400067 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.400400 24748 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:20.401191 24648 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:20.400964 24733 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.401556 24648 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.401958 24648 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:20.431573 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 47402.4us
I20260709 13:21:20.436527 16636 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.437765 24749 raft_consensus.cc:2804] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:20.438473 24749 raft_consensus.cc:697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:20.440997 24648 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.439600 24749 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:20.444195 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.444684 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.443053 24748 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.443320 24648 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.445979 24732 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:20.451858 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:20.452267 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:20.453061 16741 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:20.451928 16491 catalog_manager.cc:5697] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 21 to 22, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 22 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.454602 24748 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.465325 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.466157 16668 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 18: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:20.468214 16708 connection.cc:769] client connection to 127.15.29.1:34889 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:21:20.451431 16634 connection.cc:769] client connection to 127.15.29.1:34889 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:21:20.472123 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:20.473284 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: sendmsg error: Broken pipe (error 32). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:20.476377 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.477216 16668 raft_consensus.cc:3060] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:21:20.464293 24733 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.481770 24731 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.488653 24731 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.492049 24731 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.507022 24731 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.507772 24731 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.512032 16668 raft_consensus.cc:2468] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 28.
I20260709 13:21:20.520217 16668 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:21:20.521852 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.523278 24731 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.525602 17415 raft_consensus.cc:1275] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:21:20.527679 24749 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.528316 24749 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.529793 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.530419 16671 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 24.
I20260709 13:21:20.530646 24749 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.533217 24731 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:20.535530 24749 consensus_queue.cc:1048] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:21:20.537559 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.538096 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.538539 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.539392 24731 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:20.540359 24749 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.541009 24749 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.542181 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:20.542896 17413 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
I20260709 13:21:20.544607 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:20.548956 16708 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:20.552881 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:20.551663 24751 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:21:20.577041 24751 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.577527 24751 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:20.554673 24726 raft_consensus.cc:2804] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:21:20.582036 24726 raft_consensus.cc:697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 28 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:20.583142 24726 consensus_queue.cc:237] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.551869 24648 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.585814 24648 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.588342 24648 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.575736 24730 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.557812 24734 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:21:20.593251 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:20.595328 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.600222 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.601092 24710 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.601642 24710 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.603721 24710 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.597622 24730 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.605507 24747 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:20.605899 24730 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.606245 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:20.597559 24734 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
W20260709 13:21:20.607893 16708 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.608232 24734 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:21:20.610702 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.611327 16662 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 25.
I20260709 13:21:20.615706 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:20.616250 17374 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.616456 17416 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 17.
I20260709 13:21:20.617123 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.617918 24760 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:20.618353 24760 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.618466 24730 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.618798 24760 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:20.619323 24730 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.621734 24730 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.626354 16739 raft_consensus.cc:1275] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:21:20.628104 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:20.627894 24710 consensus_queue.cc:1048] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:21:20.629073 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:20.630082 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:20.630761 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:20.643023 24749 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.646919 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:20.641405 17374 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:20.647578 16634 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.636499 24760 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.652006 16741 raft_consensus.cc:1275] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:21:20.636579 24751 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.648483 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.649782 24648 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:20.661361 24648 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.662326 24648 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:20.663666 24760 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.665653 24751 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.666311 24730 consensus_queue.cc:1048] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:21:20.671083 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:20.671659 16634 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.672279 16634 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:20.671733 17416 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:20.679049 24749 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:20.681315 24648 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.683439 24760 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:20.684010 24648 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.689208 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.689827 16671 raft_consensus.cc:3060] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:20.689934 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:20.690260 16634 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.690740 16634 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:20.685550 24754 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.691619 24754 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:20.691890 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.690809 17415 raft_consensus.cc:3060] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:20.692759 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:20.693912 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:20.697633 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:21:20.716082 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:20.716672 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:20.699929 24734 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.692725 24710 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:20.721529 16662 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 25: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:20.726940 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:20.727566 17374 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.729019 24734 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.731102 24736 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.731765 24736 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.731634 16489 catalog_manager.cc:5697] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 27 to 28, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 28 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
W20260709 13:21:20.731699 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:20.736958 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:20.739975 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:20.741824 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:20.742250 17416 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 25: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:20.742316 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.742614 17413 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:20.745561 17415 raft_consensus.cc:2468] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:20.745961 16671 raft_consensus.cc:2468] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 26.
I20260709 13:21:20.748325 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:20.749471 17374 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.752070 24742 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.752375 24751 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:20.752805 24742 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.753422 16636 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:20.734392 24736 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.744594 16491 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 19 to 20, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 20 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:20.747702 24710 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:20.775269 24710 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.777230 24710 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:20.777585 24738 raft_consensus.cc:2804] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:20.778151 24738 raft_consensus.cc:697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:20.779052 24738 consensus_queue.cc:237] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.773860 24736 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:20.783890 24736 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.784322 24736 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:20.772469 24751 raft_consensus.cc:2804] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:21:20.787958 24751 raft_consensus.cc:697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 26 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:20.788898 24751 consensus_queue.cc:237] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:20.791746 16634 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:20.792582 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:20.792999 16634 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:20.800025 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:20.803730 16671 raft_consensus.cc:1275] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:21:20.807164 16708 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:20.808257 24747 consensus_queue.cc:1048] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:20.817184 24710 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.820258 24710 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:20.876638 24736 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.864233 24710 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.879742 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.881596 16662 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 27.
I20260709 13:21:20.882187 24754 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.882299 24710 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:20.892797 24710 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:20.884416 24742 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:20.882382 24736 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:20.900069 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 149817us
I20260709 13:21:20.914323 16491 catalog_manager.cc:5697] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 25 to 26, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 26 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:20.926488 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:20.927348 17415 raft_consensus.cc:3060] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:20.928030 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:20.928767 16739 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:20.933872 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:20.934774 16668 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:20.936416 17415 raft_consensus.cc:2468] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:20.939950 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:20.943519 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1a71b94ecfb4155a5cfe591077e9a1c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:20.944295 17415 raft_consensus.cc:3060] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:20.948724 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:20.957577 24733 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:20.958114 24733 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.958527 24733 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:20.947428 24747 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:21:20.963704 17415 raft_consensus.cc:2468] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
W20260709 13:21:20.965376 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:20.966542 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:20.944687 24736 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.965837 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:20.962149 24747 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.970072 24747 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:21:20.972378 24736 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:20.978524 24766 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:20.979110 24766 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:20.979599 24766 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:21.024981 17413 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:21.033094 16741 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:21:21.018828 16636 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:21.036317 16636 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
W20260709 13:21:21.041479 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:21.043798 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:21.042402 24732 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:21.046187 24732 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:21.048517 24732 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:21.049256 24749 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:21.053246 24765 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:21:21.050650 24768 raft_consensus.cc:2804] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:21.055505 24768 raft_consensus.cc:697] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:21:21.055583 16634 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:21.056449 24768 consensus_queue.cc:237] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:21.063822 16741 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:21.054255 24767 raft_consensus.cc:2804] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:21.106737 24733 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:21.110834 24767 raft_consensus.cc:697] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:21.112334 24767 consensus_queue.cc:237] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:21.113190 24747 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:21.121683 16634 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:21.147809 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:21.148043 24733 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:21.164402 24747 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:21.178669 24766 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:21.186606 24710 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:21.187232 24710 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:21.189546 24710 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:21.197183 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:21.216271 24766 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:21.255777 16671 raft_consensus.cc:1275] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:21:21.259423 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:21.260185 16671 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:21.270345 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:21.271497 16668 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 22: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
I20260709 13:21:21.274587 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:21.275228 16668 raft_consensus.cc:3060] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:21:21.821137 16668 consensus_meta.cc:287] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 13:21:21.821519 16668 raft_consensus.cc:2468] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 28.
W20260709 13:21:21.847446 16710 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.767s	user 0.000s	sys 0.063s
W20260709 13:21:21.847883 16710 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.000s	sys 0.064s
W20260709 13:21:21.848162 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 767544us
I20260709 13:21:21.881165 24796 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:21.910656 16491 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:34978 (request call id 242) took 1001 ms. Trace:
I20260709 13:21:21.919891 16491 rpcz_store.cc:276] 0709 13:21:20.909797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:20.909946 (+   149us) service_pool.cc:224] Handling call
0709 13:21:21.910536 (+1000590us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:20.945762 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:20.951016 (+  5254us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:20.951064 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:21:20.951085 (+    21us) tablet.cc:662] Decoding operations
0709 13:21:20.951481 (+   396us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:20.951546 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:20.951578 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:20.951830 (+   252us) tablet.cc:701] Row locks acquired
0709 13:21:20.951851 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:21:20.952010 (+   159us) write_op.cc:270] Start()
0709 13:21:20.952137 (+   127us) write_op.cc:276] Timestamp: P: 1783603280951953 usec, L: 0
0709 13:21:20.952167 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:21:20.952895 (+   728us) log.cc:844] Serialized 530 byte log entry
0709 13:21:20.971742 (+ 18847us) op_driver.cc:464] REPLICATION: finished
0709 13:21:20.977974 (+  6232us) write_op.cc:301] APPLY: starting
0709 13:21:20.978088 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:20.979273 (+  1185us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:20.979405 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:20.980328 (+   923us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:20.980562 (+   234us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:21:20.980631 (+    69us) write_op.cc:312] APPLY: finished
0709 13:21:20.980893 (+   262us) log.cc:844] Serialized 25 byte log entry
0709 13:21:20.982051 (+  1158us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:20.982133 (+    82us) write_op.cc:454] Released schema lock
0709 13:21:20.982174 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:20.982503 (+   329us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:20.982529 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":5958,"apply.run_cpu_time_us":3571,"apply.run_wall_time_us":4492,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4184,"num_ops":1,"prepare.queue_time_us":126,"prepare.run_cpu_time_us":3027,"prepare.run_wall_time_us":8942,"replication_time_us":19382}]]}
W20260709 13:21:21.936861 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:21.937391 16634 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:21.938009 16634 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:21.938634 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:21.953464 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:21.956094 24797 consensus_queue.cc:1048] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:21.962236 24748 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:21.962984 24748 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:21.964118 24797 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:21.964700 24797 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:21.965175 24797 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:21.965945 24748 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:21.979470 16634 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.004408 24736 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:22.015041 15476 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:22.014780 24797 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.022375 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.024801 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:22.026175 24804 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.026476 16739 raft_consensus.cc:2393] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 22: Already voted for candidate efa00eb5e7404570958fdebb6cd077e8 in this term.
I20260709 13:21:22.026739 24804 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.029237 24804 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.021512 24796 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:22.030277 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:22.030098 24796 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:22.030761 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.031631 24805 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:22.032146 24796 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.038015 24797 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:22.024142 16738 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
W20260709 13:21:22.073975 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 50294.1us
W20260709 13:21:22.074637 17374 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:22.075713 17374 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.083803 24760 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.084430 24760 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.086585 24760 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.091166 24760 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.091727 24760 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:22.092216 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:22.094165 24760 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.126086 24786 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:22.131664 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48339.3us
W20260709 13:21:22.132877 16634 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:22.133617 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:22.134078 16634 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.133232 24786 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:22.134904 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:22.136760 24786 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:22.138593 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 58645.5us
I20260709 13:21:22.139319 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.126834 24810 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:22.161310 16489 catalog_manager.cc:5697] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 17 to 18, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:22.164472 24808 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.167567 24808 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.170423 24808 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.174919 24805 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:22.175410 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:22.177465 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.178293 17415 raft_consensus.cc:2393] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 21: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:22.150964 24810 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.179244 24817 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:22.179723 24817 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 3c3ca2a0390b46bb9a5eda03a64eda1a)
I20260709 13:21:22.180078 24817 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:22.180739 24810 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.181286 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.183197 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.184805 24736 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.185354 24736 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.202000 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.203707 17415 raft_consensus.cc:1275] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:21:22.207113 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.207667 16668 raft_consensus.cc:1275] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:21:22.212208 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:22.198170 24805 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.216657 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:22.218092 16662 raft_consensus.cc:3060] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:22.219245 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:22.219879 16666 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
W20260709 13:21:22.220764 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 128027us
I20260709 13:21:22.221243 17415 raft_consensus.cc:1275] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:21:22.233165 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:22.241308 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.243563 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.217669 24774 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:22.245100 17374 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:22.263727 16662 raft_consensus.cc:2468] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:22.246896 24812 consensus_queue.cc:1048] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:22.247612 24799 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:22.249998 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.255040 24736 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.255801 24730 consensus_queue.cc:1048] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:22.237488 24814 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.271404 24814 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.272406 24811 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:22.273633 24814 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.226011 24805 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.276154 24805 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.276976 16668 raft_consensus.cc:1275] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:21:22.277609 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.265836 24648 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:22.279703 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.240500 24768 consensus_queue.cc:1048] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:21:22.267546 24774 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.292570 24774 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.295102 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.295977 16744 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:22.267773 24760 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:22.298588 24760 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.298735 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:22.299154 24760 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:22.307286 24791 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:22.307823 24791 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:22.308211 24791 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:22.321250 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 192364us
W20260709 13:21:22.321548 17374 leader_election.cc:336] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.328673 17374 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:22.352057 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.354413 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:22.355103 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:22.356200 17416 raft_consensus.cc:1275] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
W20260709 13:21:22.356631 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:22.362311 24795 raft_consensus.cc:2804] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:21:22.371622 24795 raft_consensus.cc:697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:22.373023 24795 consensus_queue.cc:237] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.368847 24817 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.379051 24768 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.379940 24768 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.393935 24791 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.398207 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.400746 16636 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.401903 24817 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.443176 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:22.443779 16738 raft_consensus.cc:3060] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:22.444360 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:22.446940 16708 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:22.447713 24726 raft_consensus.cc:2804] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:21:22.448235 24726 raft_consensus.cc:697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 28 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:22.449323 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:22.408172 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 63151.5us
I20260709 13:21:22.409849 24732 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.428380 24803 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.441000 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.441547 24805 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:22.442770 24774 consensus_queue.cc:1048] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:21:22.453995 16738 raft_consensus.cc:2468] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:22.429198 24768 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.455657 24726 consensus_queue.cc:237] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:22.454946 16634 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.471071 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.471909 16636 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:22.470916 16634 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.476702 16664 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Rejecting Update request from peer ba2df6a668144022a223e504ae571f21 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:22.476972 24816 raft_consensus.cc:2804] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:21:22.502921 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 48001.3us
W20260709 13:21:22.504333 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.463865 24817 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:21:22.505312 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:22.512967 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.462000 24732 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.472316 24710 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:22.508496 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.531405 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:22.532116 24807 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:22.507591 24768 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:21:22.538347 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.541941 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.545428 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:22.548566 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:22.549014 16708 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.510761 24816 raft_consensus.cc:697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:22.474507 24760 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.511467 24742 consensus_queue.cc:1048] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:21:22.509507 24796 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 13:21:22.463438 24803 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.551368 24810 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:22.564396 17376 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.565608 24731 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:22.573519 17376 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.568885 24731 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
W20260709 13:21:22.576332 16634 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.576543 24816 consensus_queue.cc:237] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.576725 24731 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.578212 24731 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:22.550817 24807 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.580788 24807 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:22.577183 16634 leader_election.cc:336] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.584121 16634 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.586769 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.587131 24805 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:22.588382 24803 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.577219 24730 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:22.593199 24809 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:22.537963 24710 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:22.628700 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.628374 17376 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.632936 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:22.612936 24809 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.637508 24731 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:22.656412 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.657127 16741 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 18.
I20260709 13:21:22.657614 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:22.657925 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:22.655206 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.642009 24791 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:22.663478 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:22.668589 16708 leader_election.cc:336] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.669030 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:22.674600 24732 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.645474 24809 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:22.677030 16634 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:22.678526 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.646700 24841 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:22.690688 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:22.693037 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:22.705755 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 77439.8us
I20260709 13:21:22.692634 24783 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:22.698846 24811 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:22.696867 24648 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:22.689885 24843 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:21:22.740363 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 62683.5us
W20260709 13:21:22.741139 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 88897.7us
I20260709 13:21:22.702535 24814 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:22.735864 24760 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.740592 24648 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:21:22.761315 16525 debug-util.cc:398] Leaking SignalData structure 0x7b080038ec60 after lost signal to thread 15479
I20260709 13:21:22.762094 24648 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:21:22.767750 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800301de0 after lost signal to thread 16528
I20260709 13:21:22.739794 24811 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:22.770419 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003c0920 after lost signal to thread 16695
W20260709 13:21:22.771981 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800160bc0 after lost signal to thread 16696
W20260709 13:21:22.773026 16525 debug-util.cc:398] Leaking SignalData structure 0x7b080023a320 after lost signal to thread 16769
W20260709 13:21:22.774015 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08001e9e00 after lost signal to thread 17446
W20260709 13:21:22.774524 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08001026c0 after lost signal to thread 24749
W20260709 13:21:22.774690 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8880 after lost signal to thread 24753
W20260709 13:21:22.775041 16525 debug-util.cc:398] Leaking SignalData structure 0x7b08003c1480 after lost signal to thread 24759
W20260709 13:21:22.775530 16525 debug-util.cc:398] Leaking SignalData structure 0x7b0800059e80 after lost signal to thread 24770
W20260709 13:21:22.775825 16525 debug-util.cc:398] Leaking SignalData structure 0x7b080031fac0 after lost signal to thread 24772
I20260709 13:21:22.784119 17414 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:22.756630 24843 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.790208 24809 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:22.783452 24814 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:22.796188 24845 log.cc:927] Time spent T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:21:22.795922 24814 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.797959 24820 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:22.812158 24839 log.cc:927] Time spent T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:21:22.827579 24849 log.cc:927] Time spent T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:21:22.800246 24820 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.851598 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.849685 24710 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.837464 24648 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:22.883719 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:22.884687 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:22.885820 17419 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:22.887818 24843 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:22.888443 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:22.884048 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:22.895031 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:22.898787 16664 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 25.
W20260709 13:21:22.899096 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.900103 24791 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:22.903962 17416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:22.906759 24791 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:22.907922 24791 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:22.909318 24811 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.910220 24820 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:22.915764 16489 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38208 (request call id 313) took 1258 ms. Trace:
I20260709 13:21:22.918185 16491 catalog_manager.cc:5697] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 21 to 22, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 22 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:22.928140 16491 catalog_manager.cc:5697] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 20 to 21, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 21 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:22.916216 16489 rpcz_store.cc:276] 0709 13:21:21.657139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:21:21.657587 (+   448us) service_pool.cc:224] Handling call
0709 13:21:22.915695 (+1258108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:21:22.260009 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:21:22.260556 (+   547us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:21:22.260589 (+    33us) write_op.cc:435] Acquired schema lock
0709 13:21:22.260606 (+    17us) tablet.cc:662] Decoding operations
0709 13:21:22.260953 (+   347us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:21:22.261030 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 13:21:22.261066 (+    36us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:21:22.261245 (+   179us) tablet.cc:701] Row locks acquired
0709 13:21:22.261261 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:21:22.261398 (+   137us) write_op.cc:270] Start()
0709 13:21:22.261504 (+   106us) write_op.cc:276] Timestamp: P: 1783603282261353 usec, L: 0
0709 13:21:22.261526 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:21:22.262213 (+   687us) log.cc:844] Serialized 530 byte log entry
0709 13:21:22.522260 (+260047us) op_driver.cc:464] REPLICATION: finished
0709 13:21:22.588473 (+ 66213us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b180033a180. stack: 00007f327cac5343 00007f327cac5240 00007f327c4541d4 00007f3286192304 00007f3286192299 00007f32859bfde1 00007f32859cc5b1 00007f327caf73f3 00007f327caf97c1 00007f327caf9779 00007f327caf9709 00007f327caf96d1 00007f327caf89cd 000000000038f354 000000000038f199 00007f327cada83a
0709 13:21:22.588595 (+   122us) write_op.cc:301] APPLY: starting
0709 13:21:22.588674 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:21:22.589785 (+  1111us) tablet.cc:1370] finished BulkCheckPresence
0709 13:21:22.589806 (+    21us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:21:22.590543 (+   737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:21:22.590731 (+   188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:21:22.590778 (+    47us) write_op.cc:312] APPLY: finished
0709 13:21:22.614007 (+ 23229us) log.cc:844] Serialized 25 byte log entry
0709 13:21:22.636249 (+ 22242us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:22.639914 (+  3665us) write_op.cc:454] Released schema lock
0709 13:21:22.641665 (+  1751us) write_op.cc:341] FINISH: Updating metrics
0709 13:21:22.663846 (+ 22181us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:21:22.668280 (+  4434us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":37551,"apply.run_cpu_time_us":3406,"apply.run_wall_time_us":63745,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8311,"num_ops":1,"prepare.queue_time_us":69588,"prepare.run_cpu_time_us":45607,"prepare.run_wall_time_us":151111,"raft.queue_time_us":74755,"raft.run_cpu_time_us":19102,"raft.run_wall_time_us":36410,"replication_time_us":260554,"spinlock_wait_cycles":14703488,"thread_start_us":250896,"threads_started":4,"wal-append.queue_time_us":183227}]]}
I20260709 13:21:22.921044 24814 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:22.938963 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:22.960350 24852 log.cc:927] Time spent T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:21:22.962618 24648 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:22.970243 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 94215.5us
W20260709 13:21:22.970935 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:22.975891 16634 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.953464 24814 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:22.978353 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:22.982445 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:22.987432 16664 raft_consensus.cc:2393] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate efa00eb5e7404570958fdebb6cd077e8 in current term 19: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:22.987432 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:22.998718 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 58926.3us
W20260709 13:21:23.015702 15476 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:23.013063 16664 raft_consensus.cc:1275] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
W20260709 13:21:23.033162 16634 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:23.040076 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:23.041015 16736 raft_consensus.cc:3060] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:23.042275 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:23.046008 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:23.046619 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:23.047240 17419 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 25.
I20260709 13:21:23.051735 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:23.054668 16736 raft_consensus.cc:2468] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 19.
I20260709 13:21:23.065766 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:23.067488 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:23.068652 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:23.069715 16636 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:23.056731 16489 catalog_manager.cc:5697] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 27 to 28, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3). New cstate: current_term: 28 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } }
I20260709 13:21:23.075912 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:23.080014 24804 consensus_queue.cc:1048] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:23.081338 24843 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:23.081858 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:23.082824 24754 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:23.083222 24754 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:23.083487 24805 raft_consensus.cc:2804] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:23.083990 24805 raft_consensus.cc:697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:23.084148 24754 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:23.084894 24805 consensus_queue.cc:237] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:23.085637 24754 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:23.083250 24730 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:23.075680 24789 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:23.095052 24789 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.095446 24789 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:21:23.097731 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.100600 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:23.104475 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:23.077824 24811 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:23.115748 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:23.117522 24710 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:23.117944 24710 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.118363 24710 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:23.119040 24730 raft_consensus.cc:2749] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:23.123230 24807 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:23.129527 24789 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:23.135288 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 57967.3us
I20260709 13:21:23.169673 24710 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:23.172211 24789 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:23.176605 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:23.177796 16664 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 in current term 26: Already voted for candidate 170921ba08b74a33995ae7c7686f8109 in this term.
W20260709 13:21:23.200228 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 51999.2us
I20260709 13:21:23.205905 24710 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:23.208774 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:23.209569 17411 raft_consensus.cc:2393] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 26: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:21:23.211979 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:23.212920 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:23.213725 24789 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:21:23.222810 16664 raft_consensus.cc:1275] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 13:21:23.232769 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:23.233322 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:23.237079 24747 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:23.239454 24843 consensus_queue.cc:1048] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:23.241364 16738 raft_consensus.cc:1275] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:21:23.244765 24821 consensus_queue.cc:1048] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:21:23.253248 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:23.255834 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:23.253780 16634 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:23.268391 16491 catalog_manager.cc:5697] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 22 to 23, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 23 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:23.269629 24805 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:21:23.302086 16487 catalog_manager.cc:5697] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 21 to 22, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 22 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.305011 16487 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 21 to 22, leader changed from ba2df6a668144022a223e504ae571f21 (127.15.29.4) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 22 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:23.350060 17411 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:21:23.353248 24821 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:23.374997 16491 catalog_manager.cc:5697] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 17 to 19, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 19 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:23.380539 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:23.448961 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:23.455274 16738 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:21:23.496407 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.550117 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:23.555542 16741 raft_consensus.cc:1275] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:21:23.568604 24710 consensus_queue.cc:1048] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:21:23.567502 16738 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:21:23.591291 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:23.586429 24805 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
W20260709 13:21:23.619493 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:23.692353 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:23.732103 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:23.763813 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:23.807066 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:23.844815 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:23.877913 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:23.880690 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:23.910843 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:23.970431 24805 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:23.971127 24805 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:23.973713 24805 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:23.975317 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:23.976074 16741 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:23.977360 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:23.980752 16634 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:23.981853 24805 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:23.982326 24805 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:23.982692 24805 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:24.008320 24805 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:24.017711 16634 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:24.018513 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:24.032012 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
W20260709 13:21:24.032603 15476 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:24.032689 16741 raft_consensus.cc:3060] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:24.032420 24805 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:24.069741 16741 raft_consensus.cc:2468] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:24.071403 16636 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:24.073122 24808 raft_consensus.cc:2804] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:24.073673 24808 raft_consensus.cc:697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:24.074502 24808 consensus_queue.cc:237] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
W20260709 13:21:24.082609 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.112136 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.155387 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:24.263737 16491 catalog_manager.cc:5697] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 21 to 23, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 23 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:24.296185 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:24.299299 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:24.331820 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:24.332901 24648 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:24.333576 24648 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:24.335914 24648 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:24.349285 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:24.350100 16744 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
I20260709 13:21:24.351649 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:24.353075 24648 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:24.353664 24648 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:24.354208 24648 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:24.361032 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:24.361826 17411 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 22: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
W20260709 13:21:24.377242 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.380852 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.393064 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.408325 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.434072 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.460244 24648 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:24.481273 24648 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:24.482456 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:24.499289 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:24.500171 17411 raft_consensus.cc:2393] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 22: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:24.504108 16738 raft_consensus.cc:1275] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:21:24.510095 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.514722 16744 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 22.
W20260709 13:21:24.605568 17374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111) [suppressed 372 similar messages]
I20260709 13:21:24.619345 24710 consensus_queue.cc:1048] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:21:24.620613 16636 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:24.622632 24710 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:24.624951 24710 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:24.626965 24710 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:24.640805 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:24.633502 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.660996 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:24.689383 16671 consensus_queue.cc:237] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:24.804205 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:24.804776 16492 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 21 to 22, leader changed from efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) to 170921ba08b74a33995ae7c7686f8109 (127.15.29.2). New cstate: current_term: 22 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:24.814039 17416 raft_consensus.cc:1275] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:21:24.840003 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.862341 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.871246 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:24.904510 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:24.912093 24648 consensus_queue.cc:1048] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:21:24.926904 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.944039 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:24.974982 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:24.996699 24805 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Committing config change with OpId 20.16: config changed from index 11 to 16, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:24.998479 24878 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:24.999326 24878 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:24.998955 17416 raft_consensus.cc:2955] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 11 to 16, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:25.003433 24878 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:25.005369 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:25.011927 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:25.012813 16738 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 22.
I20260709 13:21:25.016496 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:25.021281 24881 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:25.021701 24881 raft_consensus.cc:493] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.022023 24881 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:21:25.033456 15476 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:25.033085 16491 catalog_manager.cc:5697] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 11 to 16, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 20 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:25.047170 24881 raft_consensus.cc:515] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:25.050061 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:25.052465 24881 leader_election.cc:290] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:25.055727 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ada6718f86ea49aa96bb0fdac1acca8e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:25.056429 16741 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:25.066092 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet 894a0fdf94344f46b0426efbd37a6c40 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:52804
I20260709 13:21:25.069746 16741 raft_consensus.cc:2468] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 23.
I20260709 13:21:25.076073 17376 leader_election.cc:304] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109
I20260709 13:21:25.080292 24881 raft_consensus.cc:2804] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:25.081228 24881 raft_consensus.cc:697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:25.082389 24881 consensus_queue.cc:237] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:25.089491 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 894a0fdf94344f46b0426efbd37a6c40 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 13:21:25.101852 16741 raft_consensus.cc:1240] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 22. Current term is 23. Ops: []
I20260709 13:21:25.106662 24710 consensus_queue.cc:1059] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:21:25.107308 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:25.108053 24876 raft_consensus.cc:3055] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:25.109503 24876 raft_consensus.cc:740] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:25.110384 24876 consensus_queue.cc:260] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:25.112280 24876 raft_consensus.cc:3060] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:25.113219 24889 tablet_replica.cc:333] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:25.114737 24889 raft_consensus.cc:2243] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:25.130949 24889 raft_consensus.cc:2272] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:25.136327 24889 ts_tablet_manager.cc:1916] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:21:25.153621 16491 catalog_manager.cc:5697] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 22 to 23, leader changed from 170921ba08b74a33995ae7c7686f8109 (127.15.29.2) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 23 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:25.188743 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:25.196126 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:25.219275 24889 ts_tablet_manager.cc:1929] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.15
I20260709 13:21:25.219959 24889 log.cc:1199] T 894a0fdf94344f46b0426efbd37a6c40 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/894a0fdf94344f46b0426efbd37a6c40
I20260709 13:21:25.222113 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet 894a0fdf94344f46b0426efbd37a6c40 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
W20260709 13:21:25.306928 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:25.315006 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:25.494035 16664 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:21:25.494509 16664 pending_rounds.cc:85] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Aborting all ops after (but not including) 15
I20260709 13:21:25.494741 16664 pending_rounds.cc:107] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:21:25.495049 16664 raft_consensus.cc:2889] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:21:25.498569 24730 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:21:25.504076 16741 raft_consensus.cc:1275] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:21:25.505762 24878 consensus_queue.cc:1048] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:25.558470 24881 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:25.559247 24881 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:25.561565 24881 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:25.609155 16483 maintenance_manager.cc:419] P bfe27eaae9f4449e96031a0b1485f80c: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033380
I20260709 13:21:25.627399 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:25.629274 16664 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 26.
W20260709 13:21:25.700076 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:25.702090 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:25.705072 24730 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:21:25.705600 24730 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:25.706202 24730 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:25.840778 24730 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:25.855685 24730 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 27 election: Requested vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:25.856370 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:25.862576 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:25.879479 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:25.880156 16671 raft_consensus.cc:3060] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:25.902474 16671 raft_consensus.cc:2468] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 27.
I20260709 13:21:25.909818 17374 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: 
I20260709 13:21:25.910665 24878 raft_consensus.cc:2804] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:21:25.911235 24878 raft_consensus.cc:697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 27 LEADER]: Becoming Leader. State: Replica: ba2df6a668144022a223e504ae571f21, State: Running, Role: LEADER
I20260709 13:21:25.912230 24878 consensus_queue.cc:237] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:25.923702 17374 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:25.947991 16491 catalog_manager.cc:5697] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 24 to 27, leader changed from 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) to ba2df6a668144022a223e504ae571f21 (127.15.29.4). New cstate: current_term: 27 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:25.951427 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:25.968267 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:25.987881 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:26.010764 16482 maintenance_manager.cc:643] P bfe27eaae9f4449e96031a0b1485f80c: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.400s	user 0.036s	sys 0.016s Metrics: {"bytes_written":10350,"delete_count":0,"lbm_write_time_us":1177,"lbm_writes_lt_1ms":8,"reinsert_count":0,"update_count":396}
I20260709 13:21:26.028988 16483 maintenance_manager.cc:419] P bfe27eaae9f4449e96031a0b1485f80c: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.450020
W20260709 13:21:26.033985 15476 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:26.035352 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.089488 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:26.128332 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.160899 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:26.198109 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:26.219570 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.236173 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:26.284497 16482 maintenance_manager.cc:643] P bfe27eaae9f4449e96031a0b1485f80c: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.253s	user 0.118s	sys 0.034s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":30791,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":10009,"dirs.run_cpu_time_us":310,"dirs.run_wall_time_us":311,"lbm_read_time_us":3205,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":20,"lbm_write_time_us":16296,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":12,"peak_mem_usage":605027,"reinsert_count":0,"thread_start_us":10384,"threads_started":1,"update_count":396}
W20260709 13:21:26.326969 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:26.327922 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:26.348925 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:26.351850 17415 consensus_queue.cc:237] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:26.373538 16671 raft_consensus.cc:1275] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:21:26.376573 24893 consensus_queue.cc:1048] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:21:26.379053 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:26.381310 16671 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:26.394379 24893 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 11 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:26.409528 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:26.400386 16664 raft_consensus.cc:2955] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 11 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:26.435398 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:26.440395 16741 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:21:26.454610 24876 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:21:26.475144 16636 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3551e95f0ec5435f84de7b8f75069b13. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:26.491878 16480 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b40781eb2d304f9b86f5061ae572e1c7 with cas_config_opid_index 11: ChangeConfig:REMOVE_PEER succeeded (attempt 12)
W20260709 13:21:26.507218 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:26.512840 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:26.532846 16491 catalog_manager.cc:5697] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 11 to 14, VOTER efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3) evicted. New cstate: current_term: 19 leader_uuid: "ba2df6a668144022a223e504ae571f21" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:26.540421 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:26.553550 16671 raft_consensus.cc:1275] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Refusing update from remote peer ba2df6a668144022a223e504ae571f21: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:21:26.557577 24878 consensus_queue.cc:1048] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:21:26.562546 24710 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:21:26.569744 16738 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Committing config change with OpId 18.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:21:26.610756 16738 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:26.617091 16487 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:21:26.669269 16666 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:21:26.673071 24880 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
W20260709 13:21:26.680500 16710 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 56335fb09b2846e184d61194e1d17f74. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:26.703212 16726 tablet_service.cc:1558] Processing DeleteTablet for tablet b40781eb2d304f9b86f5061ae572e1c7 with delete_type TABLET_DATA_TOMBSTONED (TS efa00eb5e7404570958fdebb6cd077e8 not found in new config with opid_index 14) from {username='slave'} at 127.0.0.1:52804
W20260709 13:21:26.788017 24915 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.109s	user 0.002s	sys 0.003s
I20260709 13:21:26.858494 24921 tablet_replica.cc:333] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
W20260709 13:21:26.874183 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:26.901237 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:26.942378 24921 raft_consensus.cc:2243] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:26.948171 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:26.949056 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:27.004804 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 209645us
I20260709 13:21:27.008742 24921 raft_consensus.cc:2272] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:27.034569 15476 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:27.132712 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:27.140870 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:27.157964 24921 ts_tablet_manager.cc:1916] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:21:27.159873 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:27.196904 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.199851 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.204218 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:27.210602 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.253022 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 46720.4us
I20260709 13:21:27.284633 24934 ts_tablet_manager.cc:933] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:27.275219 24931 raft_consensus.cc:2955] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:21:27.303007 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.303972 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:27.309908 24934 tablet_copy_client.cc:323] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:21:27.326049 24935 ts_tablet_manager.cc:933] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:27.336825 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 3551e95f0ec5435f84de7b8f75069b13 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:21:27.337483 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 3551e95f0ec5435f84de7b8f75069b13 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-3551e95f0ec5435f84de7b8f75069b13
I20260709 13:21:27.351689 24935 tablet_copy_client.cc:323] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.3:36655
I20260709 13:21:27.353730 16681 tablet_copy_source_session.cc:215] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:21:27.354478 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.355054 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 61112.4us
W20260709 13:21:27.362847 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:27.368747 24921 ts_tablet_manager.cc:1929] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 19.13
I20260709 13:21:27.373278 24921 log.cc:1199] T b40781eb2d304f9b86f5061ae572e1c7 P efa00eb5e7404570958fdebb6cd077e8: Deleting WAL directory at /tmp/dist-test-taskIC67se/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783603014893117-15476-0/minicluster-data/ts-2-root/wals/b40781eb2d304f9b86f5061ae572e1c7
I20260709 13:21:27.376369 16480 catalog_manager.cc:5028] TS efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): tablet b40781eb2d304f9b86f5061ae572e1c7 (table test-workload [id=4a927c90303243679103bb3906337003]) successfully deleted
I20260709 13:21:27.386087 16666 raft_consensus.cc:2955] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Committing config change with OpId 15.11: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:21:27.388796 16487 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index -1 to 11, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:27.393335 24934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3551e95f0ec5435f84de7b8f75069b13. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:27.402436 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:27.435194 16668 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:27.437131 24934 tablet_copy_client.cc:806] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
W20260709 13:21:27.438259 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:27.450115 24934 tablet_copy_client.cc:670] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
W20260709 13:21:27.450229 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:27.465523 24934 tablet_copy_client.cc:538] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:21:27.476703 24934 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
I20260709 13:21:27.479198 16755 tablet_copy_service.cc:140] P efa00eb5e7404570958fdebb6cd077e8: Received BeginTabletCopySession request for tablet 56335fb09b2846e184d61194e1d17f74 from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:41680)
I20260709 13:21:27.480878 16755 tablet_copy_service.cc:161] P efa00eb5e7404570958fdebb6cd077e8: Beginning new tablet copy session on tablet 56335fb09b2846e184d61194e1d17f74 from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:41680: session id = ba2df6a668144022a223e504ae571f21-56335fb09b2846e184d61194e1d17f74
I20260709 13:21:27.505491 16755 tablet_copy_source_session.cc:215] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:21:27.521983 16636 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455): Couldn't send request to peer ba2df6a668144022a223e504ae571f21. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 98936dee7edb4aa5873923a47699a89c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:27.523125 16738 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:21:27.525321 24710 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:27.538151 24935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56335fb09b2846e184d61194e1d17f74. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:21:27.557827 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:27.561239 24908 raft_consensus.cc:2955] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Committing config change with OpId 24.15: config changed from index -1 to 15, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
W20260709 13:21:27.571727 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:27.565164 16738 raft_consensus.cc:2955] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index -1 to 15, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } } }
I20260709 13:21:27.609800 24935 tablet_copy_client.cc:806] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:21:27.622216 16487 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index -1 to 15, NON_VOTER ba2df6a668144022a223e504ae571f21 (127.15.29.4) added. New cstate: current_term: 24 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:27.625105 24935 tablet_copy_client.cc:670] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:21:27.626091 24934 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:21:27.633231 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:27.635015 24934 tablet_bootstrap.cc:492] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:21:27.635964 24934 ts_tablet_manager.cc:1403] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.160s	user 0.040s	sys 0.010s
I20260709 13:21:27.647024 24935 tablet_copy_client.cc:538] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:21:27.655128 24934 raft_consensus.cc:359] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:27.659102 24934 raft_consensus.cc:740] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:21:27.659863 24934 consensus_queue.cc:260] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:27.668155 24934 ts_tablet_manager.cc:1434] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:21:27.675141 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-3551e95f0ec5435f84de7b8f75069b13 received from {username='slave'} at 127.0.0.1:38524
I20260709 13:21:27.675596 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-3551e95f0ec5435f84de7b8f75069b13 on tablet 3551e95f0ec5435f84de7b8f75069b13 with peer ba2df6a668144022a223e504ae571f21
I20260709 13:21:27.821936 24935 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
W20260709 13:21:27.860119 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.861145 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:27.865914 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:27.873294 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:27.940039 24935 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:21:27.957880 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:27.958745 24935 tablet_bootstrap.cc:492] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:21:27.965714 24935 ts_tablet_manager.cc:1403] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.144s	user 0.033s	sys 0.011s
W20260709 13:21:27.978931 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08001cfa60 after lost signal to thread 15479
I20260709 13:21:27.978650 24935 raft_consensus.cc:359] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:27.979523 24935 raft_consensus.cc:740] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
W20260709 13:21:27.980084 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2b20 after lost signal to thread 16528
I20260709 13:21:27.980193 24935 consensus_queue.cc:260] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
W20260709 13:21:27.981500 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003c40c0 after lost signal to thread 16695
I20260709 13:21:27.982508 24935 ts_tablet_manager.cc:1434] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
W20260709 13:21:27.982667 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08003f0700 after lost signal to thread 16769
I20260709 13:21:27.987147 16755 tablet_copy_service.cc:342] P efa00eb5e7404570958fdebb6cd077e8: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-56335fb09b2846e184d61194e1d17f74 received from {username='slave'} at 127.0.0.1:41680
I20260709 13:21:27.995191 16755 tablet_copy_service.cc:434] P efa00eb5e7404570958fdebb6cd077e8: ending tablet copy session ba2df6a668144022a223e504ae571f21-56335fb09b2846e184d61194e1d17f74 on tablet 56335fb09b2846e184d61194e1d17f74 with peer ba2df6a668144022a223e504ae571f21
W20260709 13:21:27.997335 16766 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0e80 after lost signal to thread 17446
W20260709 13:21:28.035234 15476 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:28.044047 24935 ts_tablet_manager.cc:933] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: Initiating tablet copy from peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:28.079214 24935 tablet_copy_client.cc:323] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: tablet copy: Beginning tablet copy session from remote peer at address 127.15.29.2:42639
I20260709 13:21:28.082391 17411 raft_consensus.cc:1217] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 LEARNER]: Deduplicated request from leader. Original: 18.10->[18.11-18.11]   Dedup: 18.11->[]
I20260709 13:21:28.083392 16681 tablet_copy_service.cc:140] P 170921ba08b74a33995ae7c7686f8109: Received BeginTabletCopySession request for tablet 98936dee7edb4aa5873923a47699a89c from peer ba2df6a668144022a223e504ae571f21 ({username='slave'} at 127.0.0.1:38524)
I20260709 13:21:28.083978 16681 tablet_copy_service.cc:161] P 170921ba08b74a33995ae7c7686f8109: Beginning new tablet copy session on tablet 98936dee7edb4aa5873923a47699a89c from peer ba2df6a668144022a223e504ae571f21 at {username='slave'} at 127.0.0.1:38524: session id = ba2df6a668144022a223e504ae571f21-98936dee7edb4aa5873923a47699a89c
I20260709 13:21:28.092449 16681 tablet_copy_source_session.cc:215] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:21:28.105826 24935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98936dee7edb4aa5873923a47699a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:21:28.171675 24935 tablet_copy_client.cc:806] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 0 data blocks...
I20260709 13:21:28.172318 24935 tablet_copy_client.cc:670] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: tablet copy: Starting download of 1 WAL segments...
I20260709 13:21:28.178628 24935 tablet_copy_client.cc:538] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:21:28.231391 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:28.245194 17411 raft_consensus.cc:1217] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Deduplicated request from leader. Original: 15.10->[15.11-15.11]   Dedup: 15.11->[]
I20260709 13:21:28.258344 24935 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: Bootstrap starting.
W20260709 13:21:28.278997 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:28.292029 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:28.446578 24935 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=1 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:21:28.451653 24935 tablet_bootstrap.cc:492] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: Bootstrap complete.
I20260709 13:21:28.452396 24935 ts_tablet_manager.cc:1403] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: Time spent bootstrapping tablet: real 0.196s	user 0.045s	sys 0.009s
I20260709 13:21:28.460848 24935 raft_consensus.cc:359] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:28.472846 24935 raft_consensus.cc:740] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: ba2df6a668144022a223e504ae571f21, State: Initialized, Role: LEARNER
I20260709 13:21:28.473642 24935 consensus_queue.cc:260] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: NON_VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: true } }
I20260709 13:21:28.476377 24935 ts_tablet_manager.cc:1434] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.001s
I20260709 13:21:28.482410 16681 tablet_copy_service.cc:342] P 170921ba08b74a33995ae7c7686f8109: Request end of tablet copy session ba2df6a668144022a223e504ae571f21-98936dee7edb4aa5873923a47699a89c received from {username='slave'} at 127.0.0.1:38524
I20260709 13:21:28.482831 16681 tablet_copy_service.cc:434] P 170921ba08b74a33995ae7c7686f8109: ending tablet copy session ba2df6a668144022a223e504ae571f21-98936dee7edb4aa5873923a47699a89c on tablet 98936dee7edb4aa5873923a47699a89c with peer ba2df6a668144022a223e504ae571f21
W20260709 13:21:28.520579 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:28.612057 17419 raft_consensus.cc:1217] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.14->[24.15-24.15]   Dedup: 24.15->[]
W20260709 13:21:28.674238 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:28.686887 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:28.687342 24931 raft_consensus.cc:1064] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:21:28.690129 24931 consensus_queue.cc:237] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:28.702275 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:28.720959 24710 raft_consensus.cc:1064] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
I20260709 13:21:28.723521 24710 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 9, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:28.739984 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:28.740955 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:28.776005 17419 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 LEARNER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:28.781666 16668 raft_consensus.cc:1275] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:21:28.784363 17411 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:28.786580 16741 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:21:28.786330 24931 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:28.793228 24950 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:21:28.794467 24908 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:21:28.795821 24927 consensus_queue.cc:1048] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:21:28.828059 24952 log.cc:927] Time spent T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:21:28.855372 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:28.871488 24908 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Committing config change with OpId 18.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:28.880869 16741 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:28.861773 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:28.901743 17420 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Committing config change with OpId 18.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:28.899333 16634 consensus_peers.cc:597] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:28.911136 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:28.910921 16491 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:21:28.921944 24932 raft_consensus.cc:2955] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:28.977463 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 57498.7us
I20260709 13:21:28.976734 16668 raft_consensus.cc:2955] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.005784 17411 raft_consensus.cc:2955] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.016026 24710 raft_consensus.cc:1064] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: attempting to promote NON_VOTER ba2df6a668144022a223e504ae571f21 to VOTER
W20260709 13:21:29.020363 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:29.026103 24710 consensus_queue.cc:237] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 13, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:29.035750 15476 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:29.039067 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:29.044490 16487 catalog_manager.cc:5697] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: config changed from index 11 to 12, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 15 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:21:29.059046 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:21:29.069468 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:29.093035 17420 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 LEARNER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:21:29.096566 16741 raft_consensus.cc:1275] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:21:29.101573 24950 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:21:29.104563 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:29.108245 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:29.113534 24950 consensus_queue.cc:1048] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:21:29.166383 16666 consensus_queue.cc:237] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:29.205958 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:29.209538 17411 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:29.213241 24710 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:21:29.224190 24951 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.229293 17411 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.243167 24908 raft_consensus.cc:2955] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index 15 to 16, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.246106 16737 raft_consensus.cc:2955] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 15 to 16, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.248080 16741 raft_consensus.cc:1275] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:21:29.255062 24710 consensus_queue.cc:1048] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:21:29.261749 16738 raft_consensus.cc:2955] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Committing config change with OpId 18.13: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:29.284320 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47266.3us
I20260709 13:21:29.302397 16492 catalog_manager.cc:5697] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 12 to 13, VOTER 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1) evicted. New cstate: current_term: 18 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:29.319765 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:29.320870 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:29.336730 17411 raft_consensus.cc:2955] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 15 to 16, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:29.358918 16492 catalog_manager.cc:5697] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 reported cstate change: config changed from index 15 to 16, ba2df6a668144022a223e504ae571f21 (127.15.29.4) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
W20260709 13:21:29.375270 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:29.378827 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.379521 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:29.386758 16478 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3551e95f0ec5435f84de7b8f75069b13 with cas_config_opid_index 12: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:21:29.390352 16478 catalog_manager.cc:4755] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3551e95f0ec5435f84de7b8f75069b13: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:29.401031 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.441577 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.451589 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:29.464797 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:29.502655 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:29.806737 16478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111) [suppressed 454 similar messages]
W20260709 13:21:30.038980 15476 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:30.502645 16478 catalog_manager.cc:4755] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3551e95f0ec5435f84de7b8f75069b13: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:30.626164 24977 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.626896 24977 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.628947 24977 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.650059 24976 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.650966 24976 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.668754 24976 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.678105 24985 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.679095 24983 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.677495 24979 raft_consensus.cc:493] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.679679 24983 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.679883 24979 raft_consensus.cc:515] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.682796 24983 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:30.683108 24979 leader_election.cc:290] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.678751 24985 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.685663 24985 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.770102 24982 raft_consensus.cc:493] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:30.827171 24979 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.828711 24979 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.830425 24984 raft_consensus.cc:493] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:30.830775 24979 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.827459 24976 raft_consensus.cc:493] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.831925 24976 raft_consensus.cc:515] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.846679 24984 raft_consensus.cc:515] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.881995 24980 raft_consensus.cc:493] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:30.881906 24988 raft_consensus.cc:493] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.847942 24982 raft_consensus.cc:515] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.905750 24985 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.914072 24986 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.916574 24984 leader_election.cc:290] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.905259 24980 raft_consensus.cc:515] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.901541 24976 leader_election.cc:290] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.918566 24992 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.921747 24992 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.923053 24982 leader_election.cc:290] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.904349 24988 raft_consensus.cc:515] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:30.924235 24992 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.925158 24988 leader_election.cc:290] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.925225 24980 leader_election.cc:290] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.906610 24985 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.934101 24985 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.919960 24987 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
W20260709 13:21:30.970849 16634 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 301074us
I20260709 13:21:30.960726 24974 raft_consensus.cc:493] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.965107 24992 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.975339 24981 raft_consensus.cc:493] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.979434 24997 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.980348 24996 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.981376 24973 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.934949 24986 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.982513 24979 raft_consensus.cc:493] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.991118 24991 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:30.982678 24976 raft_consensus.cc:493] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.997009 24976 raft_consensus.cc:515] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.997866 24988 raft_consensus.cc:493] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.998358 24982 raft_consensus.cc:493] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.998447 24988 raft_consensus.cc:515] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:30.999190 25003 raft_consensus.cc:493] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:30.999406 24979 raft_consensus.cc:515] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.999781 25003 raft_consensus.cc:515] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.000885 24988 leader_election.cc:290] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.001499 24979 leader_election.cc:290] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.001816 25003 leader_election.cc:290] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:30.991304 24974 raft_consensus.cc:515] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.002482 24998 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.994025 24981 raft_consensus.cc:515] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.003129 24998 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.003856 24974 leader_election.cc:290] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:30.974454 24987 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.004163 24996 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.005084 24981 leader_election.cc:290] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.005257 24982 raft_consensus.cc:515] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.005581 24997 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:30.999236 24976 leader_election.cc:290] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.006485 24991 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.007472 24982 leader_election.cc:290] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.006031 24987 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.007879 24997 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.009658 24991 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.010046 25002 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.010697 24975 raft_consensus.cc:493] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:31.010743 24985 raft_consensus.cc:493] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:31.011096 24989 raft_consensus.cc:493] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:30.993381 24992 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.011813 24989 raft_consensus.cc:515] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.011859 24975 raft_consensus.cc:515] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.013827 24992 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.013991 24989 leader_election.cc:290] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:30.992762 24980 raft_consensus.cc:493] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.014266 24975 leader_election.cc:290] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.014599 24980 raft_consensus.cc:515] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.011548 24985 raft_consensus.cc:515] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.015909 24977 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.016397 24977 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.016773 24980 leader_election.cc:290] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.018337 24977 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.018615 24986 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.019169 24994 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.019330 24999 raft_consensus.cc:493] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.019708 24994 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.019796 24999 raft_consensus.cc:515] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.020759 24993 raft_consensus.cc:493] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:31.011178 25005 raft_consensus.cc:493] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:31.021301 24993 raft_consensus.cc:515] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.021757 24999 leader_election.cc:290] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.010538 25002 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.022073 25005 raft_consensus.cc:515] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.023967 24993 leader_election.cc:290] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.024212 25005 leader_election.cc:290] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.009354 24973 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.026105 24991 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.026410 24998 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.006372 24996 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.026638 24991 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.027415 24973 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.028726 24991 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.028864 24977 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:31.039636 15476 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:31.016866 24985 leader_election.cc:290] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.028968 25002 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.047320 24985 raft_consensus.cc:493] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:31.047982 24985 raft_consensus.cc:515] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.050374 24985 leader_election.cc:290] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.021790 24994 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.033203 24977 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.051265 24985 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.065575 24985 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.067199 24977 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.067768 24985 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.053686 24991 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.068509 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.068666 24991 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.053922 24992 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.071096 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79e87765622246cc907a538b35c52c64" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.071434 24992 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.073266 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.074092 24992 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.070726 24991 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.079275 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.079795 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.079998 16664 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 20.
I20260709 13:21:31.074626 24992 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.080507 16666 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 17.
I20260709 13:21:31.081147 24992 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.082826 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0ed8d0a60640a0a5ce3a772c3eef5c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.083475 24992 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.090420 24980 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.091239 24980 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.092314 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "894a0fdf94344f46b0426efbd37a6c40" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.090562 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.094514 17415 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:31.019173 24986 raft_consensus.cc:493] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.096290 24980 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.097576 24988 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.093269 24985 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.098855 24988 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.099743 24995 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.100291 24995 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.100991 24988 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.097705 24987 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.101871 24987 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.098973 24985 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.102842 24995 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.103868 24987 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.104673 24985 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.097903 24991 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.110035 24991 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.112000 24991 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:31.116245 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:31.116595 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.118624 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "173ca11ec288406aa68fe01a80d48e30" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.123134 24985 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.123016 24986 raft_consensus.cc:515] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.123904 24985 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.125490 24986 leader_election.cc:290] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.125862 24985 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.126709 24985 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.127269 24985 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.129321 24985 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.135181 24992 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.135375 24987 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.135975 24992 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.136108 24987 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.164826 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40781eb2d304f9b86f5061ae572e1c7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.166282 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.165550 17411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb360c38974b4352a4febd1e0aada45d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:31.148140 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:31.174398 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.166685 17414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.140298 24987 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.175683 17414 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 21.
I20260709 13:21:31.175777 17413 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.143432 24990 raft_consensus.cc:493] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.144073 24977 raft_consensus.cc:493] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.176930 24977 raft_consensus.cc:515] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.167366 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee99829f455541f09dac21015b00282e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.178246 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bab84fe76da941dc98416ce3f8c6f35b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.176735 24990 raft_consensus.cc:515] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.143721 24994 raft_consensus.cc:493] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.179394 24994 raft_consensus.cc:515] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.181118 24990 leader_election.cc:290] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.181758 24994 leader_election.cc:290] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.159601 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.183514 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6b98cb4c3414b4e86be2fc2bcd9e2b5" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:31.184521 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:31.185202 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:31.185976 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:31.188364 24977 leader_election.cc:290] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.139411 25011 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.189771 25011 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:31.192785 25011 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:31.186530 16634 leader_election.cc:336] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.193783 16634 leader_election.cc:336] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.194434 16634 leader_election.cc:336] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.195009 16634 leader_election.cc:336] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.195688 16634 leader_election.cc:336] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.196214 16634 leader_election.cc:336] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.196695 16634 leader_election.cc:336] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.198369 16634 leader_election.cc:336] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.198935 16634 leader_election.cc:336] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.199555 16634 leader_election.cc:336] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.200052 16634 leader_election.cc:336] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.201264 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.138832 24992 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:31.218240 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.219111 16668 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 25.
I20260709 13:21:31.219199 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.218490 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.219846 16671 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:21:31.220306 16666 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 16.
I20260709 13:21:31.220624 24973 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.220304 25013 raft_consensus.cc:493] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader ba2df6a668144022a223e504ae571f21)
I20260709 13:21:31.221288 24973 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.221527 25013 raft_consensus.cc:515] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.223446 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.223601 25013 leader_election.cc:290] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.224135 16663 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 21.
I20260709 13:21:31.228220 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9aa91533ea94ae392057ef582e0c419" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.229481 24973 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.236182 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.239485 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d02ec9cc775248dfa8321ae141d0ad37" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.240118 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:31.240950 16634 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.240959 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e96ec0851c4732b9a3e1272e9fd80f" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.242313 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.240182 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.243099 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "446649301a1c45808dccfebfadfed54c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.243407 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.245846 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.242908 16671 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2df6a668144022a223e504ae571f21 in term 18.
I20260709 13:21:31.243788 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecb93faa15cf4f7b853b030dfdc6af28" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.257716 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.259411 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.260329 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.260780 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.261418 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92129e39cb38498ea9f0addc0a33f247" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.262070 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b778f817264440987b35333e6540c72" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.263367 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.264010 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.264552 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d104b6d16beb411c9099f23901cebbb2" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.264621 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.276044 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4951b0fadf5b43e49a71f0bb4fb7a85e" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.277264 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b74fd3415d14ae3aeb38c349eca0494" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.276991 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42839b9fe9ae47a5a548348ea9b4eee1" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.277647 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f919f8e7484c4f359aedc6ac05b45076" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.279212 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.281317 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3249701d37834e80bbe408e24ced7d54" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.283510 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.291656 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73089175ee9c465197a1a11e304346be" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.292272 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.293550 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f202d4fdd58454a9c8e006ea35dbc46" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.294835 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fe018a9ac6d466983da014c7b86d36f" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.291965 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40930ae2df144d709bfb017b86dfbf2b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.299198 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92d5dbc15e7c4691b7ef572175476c3d" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.299857 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.302911 24991 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:21:31.303310 24991 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.303681 24991 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:31.312274 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5bfdfe1aa047409c9214ba1f43eb434a" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.314450 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2125e4b55933488bbb6e25dba4189adb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.314569 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3326ab1787714e708abfea6cf0509109" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.316860 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e91194e070834298ba7ca9d0bdacc3d7" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.320494 24991 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.324281 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5e5fb284dac45b2a4c12e2ec38a1784" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.325019 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81853b1e217c4b3aa8033dd7e0fdf211" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.325990 16740 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.326299 16738 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.327965 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd752e770ac43a799af500f16d446d5" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.328310 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff94f30d9bba44ee9059993ecbbbf398" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.328835 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d92a8b6dd5a4c64b749d3505ca2033e" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.329360 16736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.330459 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.331030 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.331651 16739 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.332299 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
W20260709 13:21:31.343438 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 43662.9us
I20260709 13:21:31.345747 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.346696 24992 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:31.347204 24992 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.347647 24992 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:31.348335 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98936dee7edb4aa5873923a47699a89c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.354233 16710 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:31.357183 24992 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.361383 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:31.364135 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0445daac2da44dc5af84baa8514bb195" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.364195 25009 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:31.364912 25009 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:31.365306 25009 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:31.373029 24991 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.375517 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.377137 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10731e2d513f466faf0d3f2030af5469" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.380501 24987 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:21:31.381109 24987 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.381603 24987 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:31.382905 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8243711a48fd4d8c85d79e7c10801c4f" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.385908 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f836a80f84054393b6251b24eaf890f1" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.391601 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.392830 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.393632 24986 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:31.393954 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.394052 24986 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.394698 24986 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:31.394894 24998 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:21:31.395048 24991 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:31.395396 24998 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.395474 24991 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.395877 24998 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:31.395926 24991 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:31.397614 25009 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.406981 24998 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:31.409237 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:31.415513 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:31.415757 25009 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:31.411455 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3551e95f0ec5435f84de7b8f75069b13" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:31.416920 24998 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.425180 16636 leader_election.cc:304] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.430998 24992 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.431151 17415 raft_consensus.cc:1240] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 16. Current term is 17. Ops: []
I20260709 13:21:31.413481 24986 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.433379 24986 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.434612 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.434888 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.390846 24987 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.437929 24987 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.439762 24985 raft_consensus.cc:2749] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:31.451814 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.452216 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.453119 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:31.453485 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.454469 16636 leader_election.cc:304] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:31.453372 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 59484.1us
I20260709 13:21:31.456116 17374 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.457502 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.458489 24995 raft_consensus.cc:2749] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.459913 16636 leader_election.cc:304] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.462594 24995 raft_consensus.cc:2749] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:21:31.464401 16636 leader_election.cc:304] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.465095 24986 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.465952 24986 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:31.466311 24986 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.466655 24986 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:31.477504 16708 leader_election.cc:304] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.478626 25009 raft_consensus.cc:2749] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:31.479696 24995 raft_consensus.cc:2749] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:21:31.481253 16636 leader_election.cc:304] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.481316 24991 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.482331 17374 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.482343 25009 consensus_queue.cc:1059] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:31.484524 24991 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.485399 16708 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.486320 17415 raft_consensus.cc:1240] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:31.490383 25013 raft_consensus.cc:3055] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:21:31.491178 25013 raft_consensus.cc:740] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.491427 25011 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:31.492986 25013 consensus_queue.cc:260] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.494526 25013 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:21:31.502772 24995 raft_consensus.cc:2749] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:21:31.509044 17374 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.509235 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.508275 24987 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:31.512377 17374 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:31.514443 16478 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 46259.5us
I20260709 13:21:31.515645 16636 leader_election.cc:304] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.515817 16708 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.520192 16636 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.521384 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:31.514510 24987 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:31.514288 24994 consensus_queue.cc:1059] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:21:31.517771 24983 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:31.521167 24973 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.519686 24985 raft_consensus.cc:2749] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:31.524103 24980 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:31.535296 16636 leader_election.cc:304] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.538506 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21; no voters: efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.540619 16636 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.539974 24989 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:31.542415 24989 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Starting leader election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:31.542866 24989 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:31.543656 24985 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.547778 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.548993 17409 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56335fb09b2846e184d61194e1d17f74" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.549572 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.548084 17415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:31.550635 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "374e8578b72347509cc3ea1e6f464f31" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:31.552515 16478 catalog_manager.cc:4755] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3551e95f0ec5435f84de7b8f75069b13: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.537827 24980 raft_consensus.cc:2749] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:31.557605 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:31.566568 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:31.571458 17409 raft_consensus.cc:1240] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 17. Current term is 18. Ops: []
I20260709 13:21:31.577369 16636 leader_election.cc:304] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.578707 16636 leader_election.cc:304] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:31.567230 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:31.581073 25011 raft_consensus.cc:3055] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:31.582701 25011 raft_consensus.cc:740] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.581815 24973 consensus_queue.cc:1059] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:31.584052 24973 raft_consensus.cc:3055] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:21:31.584453 24973 raft_consensus.cc:740] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.585386 24973 consensus_queue.cc:260] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.586596 24973 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:21:31.600037 16708 leader_election.cc:336] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.600570 16708 leader_election.cc:304] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.601399 24985 raft_consensus.cc:2749] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:21:31.601908 16636 leader_election.cc:304] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.602193 24985 raft_consensus.cc:2749] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:21:31.602736 25013 raft_consensus.cc:2749] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:21:31.603900 16708 leader_election.cc:336] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.604368 16708 leader_election.cc:304] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.588433 24989 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.589219 24986 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:31.591259 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.588456 25011 consensus_queue.cc:260] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.607292 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.607656 17412 raft_consensus.cc:1240] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 18. Current term is 19. Ops: []
I20260709 13:21:31.608078 25011 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:31.608136 24983 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:31.608383 17374 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.610271 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:31.610338 24986 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:31.593987 17410 raft_consensus.cc:1240] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 25. Current term is 26. Ops: []
W20260709 13:21:31.611055 17374 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.611807 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.612459 24989 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:31.612782 17374 leader_election.cc:336] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.613119 24986 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:31.613236 17374 leader_election.cc:304] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.613972 24986 raft_consensus.cc:2749] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:31.614240 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:31.622822 24995 raft_consensus.cc:2749] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:21:31.623282 17374 leader_election.cc:336] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.623842 17374 leader_election.cc:304] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.624691 24986 raft_consensus.cc:2749] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:21:31.625131 17374 leader_election.cc:336] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.627081 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.627525 25007 consensus_queue.cc:1059] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:21:31.629626 24981 raft_consensus.cc:3055] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:21:31.629536 25007 consensus_queue.cc:1059] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:21:31.630100 24981 raft_consensus.cc:740] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.630640 25007 raft_consensus.cc:3055] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:21:31.631460 25007 raft_consensus.cc:740] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.631105 24981 consensus_queue.cc:260] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.632726 24981 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:31.632375 25007 consensus_queue.cc:260] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.633967 25007 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:21:31.625535 25013 raft_consensus.cc:2749] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.624413 16636 leader_election.cc:304] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.642232 16636 leader_election.cc:304] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
W20260709 13:21:31.639954 16708 leader_election.cc:336] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.643040 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.645874 25013 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:31.647032 16708 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.647495 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.648239 24995 raft_consensus.cc:2749] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:31.648252 24989 raft_consensus.cc:2749] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:31.650369 16636 leader_election.cc:304] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.651108 24995 raft_consensus.cc:2749] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:31.651468 17374 leader_election.cc:304] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.658012 24986 raft_consensus.cc:2749] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:21:31.652155 16636 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.652745 25013 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:31.653519 16708 leader_election.cc:336] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.664270 16708 leader_election.cc:304] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:31.658284 17374 leader_election.cc:336] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.664925 17374 leader_election.cc:304] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.665025 25013 raft_consensus.cc:2749] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:31.666009 16708 leader_election.cc:336] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.666189 24986 raft_consensus.cc:2749] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:31.666883 24995 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:31.667425 16636 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.668172 24995 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:21:31.666991 17374 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.666465 16708 leader_election.cc:304] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.672600 25013 raft_consensus.cc:2749] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:31.668936 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.674472 24986 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:31.675556 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.675835 16708 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.680605 17412 raft_consensus.cc:1240] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 20. Current term is 21. Ops: []
I20260709 13:21:31.682047 17374 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.680706 16737 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:31.685127 25013 consensus_queue.cc:1059] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:21:31.686189 24986 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:31.682063 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.686228 25013 raft_consensus.cc:3055] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:21:31.687552 16708 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.687733 25013 raft_consensus.cc:740] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.688676 25007 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:31.688644 25013 consensus_queue.cc:260] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:31.689642 16708 leader_election.cc:336] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.690111 25013 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:21:31.690135 16708 leader_election.cc:304] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:31.702171 17374 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.702822 17374 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:31.703461 17443 debug-util.cc:398] Leaking SignalData structure 0x7b080034f460 after lost signal to thread 15479
W20260709 13:21:31.703909 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.704358 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:31.704658 17443 debug-util.cc:398] Leaking SignalData structure 0x7b0800362220 after lost signal to thread 16528
I20260709 13:21:31.705863 16662 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:31.706787 24983 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:31.707851 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:31.708703 16708 leader_election.cc:336] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.706092 17443 debug-util.cc:398] Leaking SignalData structure 0x7b08003316e0 after lost signal to thread 16695
W20260709 13:21:31.710180 17443 debug-util.cc:398] Leaking SignalData structure 0x7b0800323b20 after lost signal to thread 16769
W20260709 13:21:31.711447 17443 debug-util.cc:398] Leaking SignalData structure 0x7b080031df00 after lost signal to thread 17446
I20260709 13:21:31.707935 25007 raft_consensus.cc:2749] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:21:31.721352 24986 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.722947 16708 leader_election.cc:304] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.723778 25007 raft_consensus.cc:2749] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:31.724881 16708 leader_election.cc:336] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.725344 16708 leader_election.cc:304] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.727993 16666 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:31.729199 16708 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.729629 16664 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:31.729796 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.729820 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:31.729269 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:31.733345 25013 raft_consensus.cc:2749] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:21:31.736426 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:31.737282 16708 leader_election.cc:336] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.737766 16708 leader_election.cc:304] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.738667 25007 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:31.739506 25007 raft_consensus.cc:2749] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:21:31.742967 17412 raft_consensus.cc:1240] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Rejecting Update request from peer efa00eb5e7404570958fdebb6cd077e8 for earlier term 21. Current term is 22. Ops: []
I20260709 13:21:31.745031 25007 consensus_queue.cc:1059] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:31.746142 25007 raft_consensus.cc:3055] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:21:31.746491 25007 raft_consensus.cc:740] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:31.747465 25007 consensus_queue.cc:260] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.748867 25007 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:31.753458 16744 raft_consensus.cc:1240] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Rejecting Update request from peer 170921ba08b74a33995ae7c7686f8109 for earlier term 22. Current term is 23. Ops: []
W20260709 13:21:31.753723 17374 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:31.754696 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.755136 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:31.755837 24986 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:31.767792 17374 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.768391 17374 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.772981 16636 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.773983 24989 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:21:31.773980 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:31.776095 24986 raft_consensus.cc:2749] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:31.776302 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.776834 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
W20260709 13:21:31.777802 17374 leader_election.cc:336] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:31.778270 17374 leader_election.cc:304] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.779392 25013 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:21:31.794320 16636 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.795212 24989 raft_consensus.cc:2749] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.803107 24983 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:31.803365 24986 raft_consensus.cc:2749] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:21:31.806895 16636 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.808182 24995 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.809649 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
W20260709 13:21:31.809818 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:31.813830 17374 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.814635 24986 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:21:31.844131 16636 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109; no voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.846966 17374 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.847885 24995 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:31.853108 24986 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:21:31.857822 17374 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.869952 24986 raft_consensus.cc:2749] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:21:31.871336 17374 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.879040 24995 consensus_queue.cc:1059] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:21:31.880790 24989 raft_consensus.cc:3055] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:21:31.881217 24989 raft_consensus.cc:740] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
I20260709 13:21:31.882040 24989 consensus_queue.cc:260] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:31.883932 24989 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:31.895848 16668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:31.897481 24986 raft_consensus.cc:2749] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:21:31.929407 17374 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.941175 24986 raft_consensus.cc:2749] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:21:31.958604 17374 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:31.959455 24991 raft_consensus.cc:2749] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:21:31.981580 17374 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:31.989254 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:31.995078 24986 raft_consensus.cc:2749] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:21:32.011260 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:32.040222 15476 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:32.053640 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.083364 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:32.210085 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.237568 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.294217 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:32.353148 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:32.379920 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:32.542519 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:32.564757 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:32.633540 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.809571 16478 catalog_manager.cc:4755] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3551e95f0ec5435f84de7b8f75069b13: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:32.897370 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 171912us
W20260709 13:21:32.956329 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:32.957935 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:32.960356 25013 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:32.960970 25013 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:32.967250 25013 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:32.969772 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:32.984815 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:32.994921 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.001260 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:33.014998 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 134353us
I20260709 13:21:33.019425 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.020108 17412 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 21.
I20260709 13:21:33.020788 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
W20260709 13:21:33.008466 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:33.024957 25013 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.025664 25013 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.033587 25013 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.033805 25019 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.034665 25019 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.038533 25019 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:33.040819 15476 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:33.042883 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:33.043198 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.043696 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:33.043696 17410 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 26.
I20260709 13:21:33.045877 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.046564 16663 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 25.
I20260709 13:21:33.056025 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.056828 16669 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:33.053629 16710 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:33.061091 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:33.065798 25013 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:21:33.066252 25013 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.066604 25013 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:33.076809 25013 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.065249 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.079249 25007 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:21:33.079700 25007 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.080070 25007 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:33.083760 16708 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.092764 25024 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:33.093274 25024 raft_consensus.cc:493] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.093640 25024 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.107038 25007 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.109256 25007 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.117715 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:33.118510 16669 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Advancing to term 27
I20260709 13:21:33.128417 25013 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.132282 16669 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 27.
W20260709 13:21:33.135854 17376 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.3:36655, user_credentials={real_user=slave}} blocked reactor thread for 83231.2us
I20260709 13:21:33.138324 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.139258 17410 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:33.139791 16708 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:33.141847 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 81614.7us
I20260709 13:21:33.141983 25019 raft_consensus.cc:2804] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:21:33.142625 25019 raft_consensus.cc:697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 27 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.143510 25019 consensus_queue.cc:237] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.145903 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b20d43e167554c06a6ae722c1a39c5d3" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.146701 17410 raft_consensus.cc:3060] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:21:33.147723 25024 raft_consensus.cc:515] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.149967 25024 leader_election.cc:290] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:33.152379 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:33.153051 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:21:33.153555 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.154266 17412 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:21:33.171507 17412 raft_consensus.cc:2468] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:33.172387 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:33.176867 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.177279 17410 raft_consensus.cc:2468] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 27.
I20260709 13:21:33.177585 17412 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.179113 16710 leader_election.cc:304] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.182142 25019 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.212847 25019 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.214977 25019 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.215513 25019 raft_consensus.cc:2804] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:21:33.184401 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3386b3f36f454834aaf359817c1adadb" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
W20260709 13:21:33.193300 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:21:33.216142 16669 raft_consensus.cc:3060] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.219496 17412 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:33.220862 16710 leader_election.cc:304] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.221926 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.222651 17412 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:33.222898 25024 raft_consensus.cc:2804] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:33.223441 25024 raft_consensus.cc:697] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.224275 25024 consensus_queue.cc:237] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.227306 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.227845 25019 raft_consensus.cc:697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.228885 25013 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.233732 25013 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.235975 25013 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.237088 25019 consensus_queue.cc:237] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.238273 25007 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:21:33.239511 25029 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.244360 25007 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.244891 25007 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:33.244547 25029 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.247009 25029 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.249137 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.249991 17410 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:33.251328 16710 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.252293 25029 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:21:33.252838 25029 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.232721 16669 raft_consensus.cc:2468] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:33.241366 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.254070 17412 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 17.
I20260709 13:21:33.257732 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.255270 16710 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.263360 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.264147 16669 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 16.
I20260709 13:21:33.253357 25029 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:33.264830 16487 catalog_manager.cc:5697] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.268334 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.269555 16669 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 22.
I20260709 13:21:33.270454 24982 raft_consensus.cc:493] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:33.283984 24982 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.289579 25025 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:33.267105 24980 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.296761 16744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:33.290522 25025 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.303414 24980 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.307263 24980 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.310740 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:33.275573 25024 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:21:33.315358 16745 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 23.
I20260709 13:21:33.315407 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
W20260709 13:21:33.318747 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:33.320830 24982 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:33.322227 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 67086us
I20260709 13:21:33.322786 25025 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.316850 25024 raft_consensus.cc:493] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.331740 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.337620 25024 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:33.339672 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.340864 17410 raft_consensus.cc:2393] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 in current term 18: Already voted for candidate ba2df6a668144022a223e504ae571f21 in this term.
I20260709 13:21:33.343897 25025 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:21:33.344596 25025 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.345379 25025 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:33.314818 25007 raft_consensus.cc:515] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.347373 16744 raft_consensus.cc:2368] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:21:33.352967 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:33.354439 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2dc3ee5aa5844e5e9c7e3f566e36a561" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.358336 16636 leader_election.cc:400] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:21:33.360390 16636 leader_election.cc:403] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:33.361070 17410 raft_consensus.cc:2368] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 21. Current term is 22.
W20260709 13:21:33.365185 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 96074.1us
I20260709 13:21:33.365777 16636 leader_election.cc:400] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 21. Current term is 22.
I20260709 13:21:33.366215 16636 leader_election.cc:403] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:33.372442 24982 raft_consensus.cc:3060] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:21:33.350620 25024 raft_consensus.cc:515] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.370602 24975 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:21:33.382568 25024 leader_election.cc:290] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
W20260709 13:21:33.387264 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:33.393726 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:33.395967 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:33.396821 16669 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:21:33.399065 16663 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:33.400792 25029 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.396826 25007 leader_election.cc:290] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.413358 16671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:33.414103 16671 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:33.414237 16669 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
W20260709 13:21:33.417994 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:33.420174 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:33.424546 16671 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 23.
I20260709 13:21:33.431819 16708 leader_election.cc:304] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.432682 25013 raft_consensus.cc:2804] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:21:33.432868 16708 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.433348 25013 raft_consensus.cc:697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.434372 25013 consensus_queue.cc:237] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.434659 25024 raft_consensus.cc:2804] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:21:33.435781 25024 raft_consensus.cc:697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 23 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.436746 25024 consensus_queue.cc:237] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.388895 25025 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.382766 24982 raft_consensus.cc:2749] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455) with higher term. Message: Invalid argument: T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 21. Current term is 22.
I20260709 13:21:33.440779 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04d635b36e854e4f9ce8b973de8edafe" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.447667 17412 raft_consensus.cc:3060] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:21:33.444758 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f71532fa5e704f9590387cd12f98666b" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.451560 17410 raft_consensus.cc:3060] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:21:33.455711 24975 raft_consensus.cc:2749] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 170921ba08b74a33995ae7c7686f8109 for earlier term 18. Current term is 19.
I20260709 13:21:33.459748 25025 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Requested vote from peers efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.464041 25023 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:21:33.464912 25023 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.467334 25023 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.472427 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8"
I20260709 13:21:33.473308 16745 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:21:33.475207 16663 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:33.481180 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "170921ba08b74a33995ae7c7686f8109" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.482012 17412 raft_consensus.cc:2468] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:33.484928 17410 raft_consensus.cc:2468] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:33.487464 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.488399 16669 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 18.
I20260709 13:21:33.475612 25029 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.487401 16710 leader_election.cc:304] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
W20260709 13:21:33.494027 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:33.495368 17419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:33.498726 17412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04867b8a53e645f0b58c55ecec0dac8c" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.499675 17412 raft_consensus.cc:3060] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:21:33.496963 25029 raft_consensus.cc:2804] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:21:33.517402 25029 raft_consensus.cc:697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.518501 25029 consensus_queue.cc:237] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.496104 17419 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 19.
I20260709 13:21:33.524333 16745 raft_consensus.cc:2468] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 170921ba08b74a33995ae7c7686f8109 in term 24.
I20260709 13:21:33.525336 16710 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba2df6a668144022a223e504ae571f21, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.526536 25007 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:21:33.527065 25007 raft_consensus.cc:493] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:21:33.527493 25007 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.540282 24986 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:33.541087 24986 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.545810 24986 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:33.545888 25007 raft_consensus.cc:515] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.545351 16745 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "efa00eb5e7404570958fdebb6cd077e8" is_pre_election: true
I20260709 13:21:33.547773 16745 raft_consensus.cc:2368] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 23. Current term is 24.
I20260709 13:21:33.554347 17376 leader_election.cc:400] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 23. Current term is 24.
I20260709 13:21:33.554766 17376 leader_election.cc:403] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:21:33.562029 25031 raft_consensus.cc:3060] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Advancing to term 24
I20260709 13:21:33.566587 16636 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: ba2df6a668144022a223e504ae571f21
I20260709 13:21:33.566864 17412 raft_consensus.cc:2468] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 23.
I20260709 13:21:33.570076 24975 raft_consensus.cc:2804] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:21:33.570746 24975 raft_consensus.cc:697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Becoming Leader. State: Replica: 170921ba08b74a33995ae7c7686f8109, State: Running, Role: LEADER
W20260709 13:21:33.571615 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:33.571683 24975 consensus_queue.cc:237] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.584367 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "ba2df6a668144022a223e504ae571f21" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109" is_pre_election: true
I20260709 13:21:33.586062 17410 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21"
I20260709 13:21:33.586776 17410 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:21:33.591363 25007 leader_election.cc:290] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Requested vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:33.597527 16669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0b4dc6fa23d44edb449d1be0842b184" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "170921ba08b74a33995ae7c7686f8109"
I20260709 13:21:33.598245 16669 raft_consensus.cc:3060] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:21:33.606195 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:33.611539 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.612099 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:33.619874 17419 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:33.621894 16669 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:33.621937 25036 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:33.625622 25031 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655) with higher term. Message: Invalid argument: T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba2df6a668144022a223e504ae571f21 for earlier term 23. Current term is 24.
I20260709 13:21:33.636435 17410 raft_consensus.cc:2468] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efa00eb5e7404570958fdebb6cd077e8 in term 20.
I20260709 13:21:33.636974 16708 leader_election.cc:304] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8; no voters: 
I20260709 13:21:33.648722 17420 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:21:33.647075 25029 raft_consensus.cc:2804] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:21:33.653530 25036 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 13:21:33.656370 25029 raft_consensus.cc:697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Becoming Leader. State: Replica: efa00eb5e7404570958fdebb6cd077e8, State: Running, Role: LEADER
I20260709 13:21:33.664058 25029 consensus_queue.cc:237] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:33.680173 16669 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:21:33.684453 25036 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:21:33.719818 16491 catalog_manager.cc:5697] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } } }
I20260709 13:21:33.725612 16492 catalog_manager.cc:5697] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "170921ba08b74a33995ae7c7686f8109" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:21:33.735423 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:33.744558 16663 raft_consensus.cc:1275] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:21:33.748689 25023 consensus_queue.cc:1048] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:33.748973 16669 raft_consensus.cc:1275] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 13:21:33.750844 25036 consensus_queue.cc:1048] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 13:21:33.752067 16692 debug-util.cc:398] Leaking SignalData structure 0x7b080032e020 after lost signal to thread 15479
W20260709 13:21:33.753322 16692 debug-util.cc:398] Leaking SignalData structure 0x7b080036e120 after lost signal to thread 16528
W20260709 13:21:33.754845 16692 debug-util.cc:398] Leaking SignalData structure 0x7b080032b120 after lost signal to thread 16695
W20260709 13:21:33.756321 16692 debug-util.cc:398] Leaking SignalData structure 0x7b08001ba740 after lost signal to thread 16769
W20260709 13:21:33.757570 16692 debug-util.cc:398] Leaking SignalData structure 0x7b0800410f00 after lost signal to thread 17446
I20260709 13:21:33.766525 17420 raft_consensus.cc:1275] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 13:21:33.775640 25007 consensus_queue.cc:1048] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:21:33.821623 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:21:33.826534 16487 catalog_manager.cc:5697] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.841949 16487 catalog_manager.cc:5697] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.843961 16487 catalog_manager.cc:5697] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.851181 16487 catalog_manager.cc:5697] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.857161 16487 catalog_manager.cc:5697] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "efa00eb5e7404570958fdebb6cd077e8" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:21:33.866464 17419 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:33.868619 25007 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:21:33.873427 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:33.874445 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:21:33.882757 16671 raft_consensus.cc:1275] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:21:33.884680 25029 consensus_queue.cc:1048] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:33.961433 16668 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:33.972074 25007 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:34.027103 16745 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:21:34.034960 17419 raft_consensus.cc:1275] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:21:34.036293 25027 consensus_queue.cc:1048] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:21:34.041357 15476 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 16617)
I20260709 13:21:34.043138 25025 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
W20260709 13:21:34.071473 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:34.071846 17420 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:21:34.075548 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:34.076483 25023 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:21:34.078449 17419 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:34.094596 25027 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:21:34.102128 16668 raft_consensus.cc:1275] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:21:34.104038 25029 consensus_queue.cc:1048] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:21:34.116734 17410 raft_consensus.cc:1275] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Refusing update from remote peer 170921ba08b74a33995ae7c7686f8109: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:21:34.169289 25025 consensus_queue.cc:1048] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 13:21:34.210510 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:34.216737 16663 raft_consensus.cc:1275] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Refusing update from remote peer efa00eb5e7404570958fdebb6cd077e8: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:21:34.236528 25027 consensus_queue.cc:1048] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
W20260709 13:21:34.265290 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.300228 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:34.302753 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 47681.5us
W20260709 13:21:34.444123 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:34.534137 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.692679 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:34.693331 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:34.693773 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:34.859439 17374 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111) [suppressed 370 similar messages]
W20260709 13:21:35.024128 16636 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 92757.1us
W20260709 13:21:35.038362 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.041985 15476 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:35.049373 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:35.055842 16710 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.4:41455, user_credentials={real_user=slave}} blocked reactor thread for 124742us
W20260709 13:21:35.076494 16478 catalog_manager.cc:4755] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3551e95f0ec5435f84de7b8f75069b13: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:35.166724 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:35.212423 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.212963 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.227743 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.333626 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:35.334156 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:35.365784 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:35.429829 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:35.443430 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:35.471297 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:35.471801 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:21:35.526306 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:35.548450 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:35.558797 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:35.607739 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:35.656644 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.673874 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.697400 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.759178 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:35.854456 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:35.950682 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.951894 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:35.983960 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:36.006168 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:36.042584 15476 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:36.069772 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:36.134891 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:36.219038 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:36.299415 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:36.356266 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:36.376240 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.428231 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:36.509152 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:36.647917 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.737819 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:36.953326 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:36.954295 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:37.043170 15476 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:37.092710 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.101099 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:37.133107 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.133920 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.183776 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.184360 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.191248 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:37.418263 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:37.468842 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:37.593118 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.622225 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.719875 16634 consensus_peers.cc:597] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:37.820865 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:37.928000 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:38.016193 16634 consensus_peers.cc:597] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:38.022149 16634 consensus_peers.cc:597] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.043669 15476 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:38.066704 16634 consensus_peers.cc:597] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.098253 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:38.124311 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:38.129367 16634 consensus_peers.cc:597] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.149012 16634 consensus_peers.cc:597] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.193257 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:38.238097 16634 consensus_peers.cc:597] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.296165 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:21:38.328362 16634 consensus_peers.cc:597] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.355226 16634 consensus_peers.cc:597] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.355960 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:38.421190 16708 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.1:34889, user_credentials={real_user=slave}} blocked reactor thread for 59152.9us
W20260709 13:21:38.443830 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:21:38.474529 16634 consensus_peers.cc:597] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.503568 17374 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.15.29.2:42639, user_credentials={real_user=slave}} blocked reactor thread for 52068.6us
W20260709 13:21:38.548113 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.560904 16634 consensus_peers.cc:597] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.570225 16708 consensus_peers.cc:597] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:38.580466 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:38.618853 16634 consensus_peers.cc:597] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:38.664906 16634 consensus_peers.cc:597] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.673395 16634 consensus_peers.cc:597] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.712893 16634 consensus_peers.cc:597] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:38.755357 16634 consensus_peers.cc:597] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:38.785898 16634 consensus_peers.cc:597] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:38.929299 16708 consensus_peers.cc:597] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:38.964259 16708 consensus_peers.cc:597] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:39.044199 15476 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 16617)
W20260709 13:21:39.075280 16708 consensus_peers.cc:597] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:39.135278 16708 consensus_peers.cc:597] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:39.162091 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:39.207249 16478 catalog_manager.cc:4755] TS 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 3551e95f0ec5435f84de7b8f75069b13: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:39.219908 16634 consensus_peers.cc:597] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:21:39.456845 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:39.476223 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:39.513734 16708 consensus_peers.cc:597] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:39.536372 16634 consensus_peers.cc:597] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:21:39.567888 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:39.681010 15476 tablet_server.cc:196] TabletServer@127.15.29.1:0 shutdown complete.
W20260709 13:21:39.768743 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:39.818854 16708 consensus_peers.cc:597] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:21:39.836087 15476 tablet_server.cc:179] TabletServer@127.15.29.2:0 shutting down...
W20260709 13:21:39.847631 16708 consensus_peers.cc:597] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.862974 16708 consensus_peers.cc:597] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.882606 16634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111) [suppressed 455 similar messages]
I20260709 13:21:39.902617 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:39.903906 15476 tablet_replica.cc:333] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.904990 15476 raft_consensus.cc:2243] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.906002 15476 raft_consensus.cc:2272] T 04867b8a53e645f0b58c55ecec0dac8c P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.909298 15476 tablet_replica.cc:333] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.910266 15476 raft_consensus.cc:2243] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.911425 15476 raft_consensus.cc:2272] T 42839b9fe9ae47a5a548348ea9b4eee1 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.914489 15476 tablet_replica.cc:333] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
W20260709 13:21:39.915014 17374 consensus_peers.cc:597] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.915495 15476 raft_consensus.cc:2243] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.916296 15476 raft_consensus.cc:2272] T e91194e070834298ba7ca9d0bdacc3d7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.919413 15476 tablet_replica.cc:333] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.920329 15476 raft_consensus.cc:2243] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.921154 15476 raft_consensus.cc:2272] T b40781eb2d304f9b86f5061ae572e1c7 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.924106 15476 tablet_replica.cc:333] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.924930 15476 raft_consensus.cc:2243] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.926100 15476 raft_consensus.cc:2272] T 10731e2d513f466faf0d3f2030af5469 P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.929005 15476 tablet_replica.cc:333] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.929826 15476 raft_consensus.cc:2243] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.930998 15476 raft_consensus.cc:2272] T 3551e95f0ec5435f84de7b8f75069b13 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.933678 15476 tablet_replica.cc:333] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.934631 15476 raft_consensus.cc:2243] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.936642 15476 raft_consensus.cc:2272] T f919f8e7484c4f359aedc6ac05b45076 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.939783 15476 tablet_replica.cc:333] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.940672 15476 raft_consensus.cc:2243] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.941919 15476 raft_consensus.cc:2272] T 3326ab1787714e708abfea6cf0509109 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.944989 15476 tablet_replica.cc:333] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
W20260709 13:21:39.945180 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:39.945921 15476 raft_consensus.cc:2243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.947103 15476 raft_consensus.cc:2272] T 7c74caaa7fe045f9a0f6ff3b3df97602 P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.950419 15476 tablet_replica.cc:333] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
W20260709 13:21:39.951339 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.951440 15476 raft_consensus.cc:2243] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.953627 15476 raft_consensus.cc:2272] T 374e8578b72347509cc3ea1e6f464f31 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.956971 16708 consensus_peers.cc:597] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:39.961284 16708 consensus_peers.cc:597] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.963110 15476 tablet_replica.cc:333] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.963810 15476 raft_consensus.cc:2243] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.964462 15476 raft_consensus.cc:2272] T 81853b1e217c4b3aa8033dd7e0fdf211 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.966776 15476 tablet_replica.cc:333] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.967478 15476 raft_consensus.cc:2243] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.968652 15476 raft_consensus.cc:2272] T 894a0fdf94344f46b0426efbd37a6c40 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.971500 15476 tablet_replica.cc:333] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.972241 15476 raft_consensus.cc:2243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.972913 15476 raft_consensus.cc:2272] T 4951b0fadf5b43e49a71f0bb4fb7a85e P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.975430 15476 tablet_replica.cc:333] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.976286 15476 raft_consensus.cc:2243] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.977923 15476 raft_consensus.cc:2272] T 98936dee7edb4aa5873923a47699a89c P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.980602 15476 tablet_replica.cc:333] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.981308 15476 raft_consensus.cc:2243] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:39.982426 15476 raft_consensus.cc:2272] T e8b0d179ad714b59b34ab1dc57a57c02 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:39.985131 15476 tablet_replica.cc:333] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.985896 15476 raft_consensus.cc:2243] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.986502 15476 raft_consensus.cc:2272] T 9f202d4fdd58454a9c8e006ea35dbc46 P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.988302 17374 consensus_peers.cc:597] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.989378 15476 tablet_replica.cc:333] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
W20260709 13:21:39.990338 17374 consensus_peers.cc:597] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:39.990347 15476 raft_consensus.cc:2243] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.992324 15476 raft_consensus.cc:2272] T 9fe018a9ac6d466983da014c7b86d36f P 170921ba08b74a33995ae7c7686f8109 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.993142 16634 consensus_peers.cc:597] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:39.996161 15476 tablet_replica.cc:333] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:39.998301 15476 raft_consensus.cc:2243] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:39.999411 15476 raft_consensus.cc:2272] T 73089175ee9c465197a1a11e304346be P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:39.999624 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.001855 15476 tablet_replica.cc:333] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.002521 15476 raft_consensus.cc:2243] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.003131 15476 raft_consensus.cc:2272] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.005236 15476 tablet_replica.cc:333] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.006053 15476 raft_consensus.cc:2243] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.006636 15476 raft_consensus.cc:2272] T b20d43e167554c06a6ae722c1a39c5d3 P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.009234 15476 tablet_replica.cc:333] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.009994 15476 raft_consensus.cc:2243] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.011008 15476 raft_consensus.cc:2272] T a931fb289f824ff28e993f8590e64b69 P 170921ba08b74a33995ae7c7686f8109 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.013703 15476 tablet_replica.cc:333] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.014494 15476 raft_consensus.cc:2243] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.015645 15476 raft_consensus.cc:2272] T 51e96ec0851c4732b9a3e1272e9fd80f P 170921ba08b74a33995ae7c7686f8109 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.016959 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.018247 15476 tablet_replica.cc:333] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.019069 15476 raft_consensus.cc:2243] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.019757 15476 raft_consensus.cc:2272] T ada6718f86ea49aa96bb0fdac1acca8e P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.022377 15476 tablet_replica.cc:333] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.023036 15476 raft_consensus.cc:2243] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.024575 15476 raft_consensus.cc:2272] T e5e5fb284dac45b2a4c12e2ec38a1784 P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.026443 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.027571 15476 tablet_replica.cc:333] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.028329 15476 raft_consensus.cc:2243] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.028980 15476 raft_consensus.cc:2272] T 40930ae2df144d709bfb017b86dfbf2b P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.031579 15476 tablet_replica.cc:333] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.032230 15476 raft_consensus.cc:2243] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.033272 15476 raft_consensus.cc:2272] T 5bfdfe1aa047409c9214ba1f43eb434a P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.036132 15476 tablet_replica.cc:333] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.036763 15476 raft_consensus.cc:2243] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.037421 15476 raft_consensus.cc:2272] T bab84fe76da941dc98416ce3f8c6f35b P 170921ba08b74a33995ae7c7686f8109 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.041286 15476 tablet_replica.cc:333] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
W20260709 13:21:40.041685 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.042135 15476 raft_consensus.cc:2243] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:21:40.044044 16708 consensus_peers.cc:597] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.043376 15476 raft_consensus.cc:2272] T 446649301a1c45808dccfebfadfed54c P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.046339 16708 consensus_peers.cc:597] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.047181 15476 tablet_replica.cc:333] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.048023 15476 raft_consensus.cc:2243] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.048655 15476 raft_consensus.cc:2272] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.052361 15476 tablet_replica.cc:333] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.053232 15476 raft_consensus.cc:2243] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.053901 15476 raft_consensus.cc:2272] T f71532fa5e704f9590387cd12f98666b P 170921ba08b74a33995ae7c7686f8109 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.057071 15476 tablet_replica.cc:333] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.057909 15476 raft_consensus.cc:2243] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.058521 15476 raft_consensus.cc:2272] T e0b4dc6fa23d44edb449d1be0842b184 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.061314 15476 tablet_replica.cc:333] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.062086 15476 raft_consensus.cc:2243] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.063031 15476 raft_consensus.cc:2272] T 56335fb09b2846e184d61194e1d17f74 P 170921ba08b74a33995ae7c7686f8109 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.065748 15476 tablet_replica.cc:333] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.066525 15476 raft_consensus.cc:2243] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.067701 15476 raft_consensus.cc:2272] T f1a71b94ecfb4155a5cfe591077e9a1c P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.070324 15476 tablet_replica.cc:333] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.071158 15476 raft_consensus.cc:2243] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.073235 15476 raft_consensus.cc:2272] T f836a80f84054393b6251b24eaf890f1 P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.075985 15476 tablet_replica.cc:333] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.077695 15476 raft_consensus.cc:2243] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.078877 15476 raft_consensus.cc:2272] T ff94f30d9bba44ee9059993ecbbbf398 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.081622 15476 tablet_replica.cc:333] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.082428 15476 raft_consensus.cc:2243] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.083123 15476 raft_consensus.cc:2272] T bb360c38974b4352a4febd1e0aada45d P 170921ba08b74a33995ae7c7686f8109 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.085949 15476 tablet_replica.cc:333] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.086710 15476 raft_consensus.cc:2243] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.088155 15476 raft_consensus.cc:2272] T ee99829f455541f09dac21015b00282e P 170921ba08b74a33995ae7c7686f8109 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.090833 15476 tablet_replica.cc:333] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.091670 15476 raft_consensus.cc:2243] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.092842 15476 raft_consensus.cc:2272] T 9b74fd3415d14ae3aeb38c349eca0494 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.095924 15476 tablet_replica.cc:333] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
W20260709 13:21:40.094779 16708 consensus_peers.cc:597] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:21:40.096807 15476 raft_consensus.cc:2243] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:40.097337 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.100054 15476 raft_consensus.cc:2272] T 3386b3f36f454834aaf359817c1adadb P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.100536 16708 consensus_peers.cc:597] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.104736 15476 tablet_replica.cc:333] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.105691 15476 raft_consensus.cc:2243] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.107126 15476 raft_consensus.cc:2272] T abd752e770ac43a799af500f16d446d5 P 170921ba08b74a33995ae7c7686f8109 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.110080 15476 tablet_replica.cc:333] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.110935 15476 raft_consensus.cc:2243] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.112186 15476 raft_consensus.cc:2272] T 6d92a8b6dd5a4c64b749d3505ca2033e P 170921ba08b74a33995ae7c7686f8109 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.114854 15476 tablet_replica.cc:333] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.115636 15476 raft_consensus.cc:2243] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.116822 15476 raft_consensus.cc:2272] T 8243711a48fd4d8c85d79e7c10801c4f P 170921ba08b74a33995ae7c7686f8109 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:40.119740 15476 tablet_replica.cc:333] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.120461 15476 raft_consensus.cc:2243] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.121119 15476 raft_consensus.cc:2272] T 92d5dbc15e7c4691b7ef572175476c3d P 170921ba08b74a33995ae7c7686f8109 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.121320 16708 consensus_peers.cc:597] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:40.123903 15476 tablet_replica.cc:333] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.124680 15476 raft_consensus.cc:2243] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:40.125346 15476 raft_consensus.cc:2272] T 04d635b36e854e4f9ce8b973de8edafe P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.125682 16708 consensus_peers.cc:597] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:21:40.127902 15476 tablet_replica.cc:333] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.128818 15476 raft_consensus.cc:2243] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.129909 15476 raft_consensus.cc:2272] T 2125e4b55933488bbb6e25dba4189adb P 170921ba08b74a33995ae7c7686f8109 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.132011 16708 consensus_peers.cc:597] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:21:40.132748 15476 tablet_replica.cc:333] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109: stopping tablet replica
I20260709 13:21:40.133440 15476 raft_consensus.cc:2243] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:40.134517 15476 raft_consensus.cc:2272] T 0445daac2da44dc5af84baa8514bb195 P 170921ba08b74a33995ae7c7686f8109 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:40.176210 16708 consensus_peers.cc:597] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.181620 16708 consensus_peers.cc:597] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.203375 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.219266 16708 consensus_peers.cc:597] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.234167 16708 consensus_peers.cc:597] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.233933 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.405990 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Couldn't send request to peer 170921ba08b74a33995ae7c7686f8109. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:40.482897 17374 consensus_peers.cc:597] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:40.542213 17374 consensus_peers.cc:597] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:21:40.577500 17374 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:40.620293 17374 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:21:40.830744 15476 tablet_server.cc:196] TabletServer@127.15.29.2:0 shutdown complete.
W20260709 13:21:40.853876 16708 consensus_peers.cc:597] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:21:40.873416 16708 consensus_peers.cc:597] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:21:40.957510 15476 tablet_server.cc:179] TabletServer@127.15.29.3:0 shutting down...
W20260709 13:21:40.961606 17374 consensus_peers.cc:597] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:21:41.001012 17376 consensus_peers.cc:597] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.007400 25092 raft_consensus.cc:493] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:41.010007 17376 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:21:41.015005 16708 consensus_peers.cc:597] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:21:41.008196 25092 raft_consensus.cc:515] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:41.019472 25092 leader_election.cc:290] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:41.035897 16708 leader_election.cc:336] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.040244 16708 leader_election.cc:336] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.040421 25092 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.041136 16708 leader_election.cc:304] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.041285 25092 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.041961 25077 raft_consensus.cc:2749] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:41.045646 17420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a931fb289f824ff28e993f8590e64b69" candidate_uuid: "efa00eb5e7404570958fdebb6cd077e8" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "ba2df6a668144022a223e504ae571f21" is_pre_election: true
I20260709 13:21:41.048175 25092 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), ba2df6a668144022a223e504ae571f21 (127.15.29.4:41455)
I20260709 13:21:41.050951 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:41.053758 15476 tablet_replica.cc:333] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
W20260709 13:21:41.054263 16708 leader_election.cc:336] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.054733 15476 raft_consensus.cc:2243] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.054701 16708 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, ba2df6a668144022a223e504ae571f21
I20260709 13:21:41.055980 15476 raft_consensus.cc:2272] T e91194e070834298ba7ca9d0bdacc3d7 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.059084 15476 tablet_replica.cc:333] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.060148 25092 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:41.060916 15476 raft_consensus.cc:2243] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.062891 15476 raft_consensus.cc:2272] T 04867b8a53e645f0b58c55ecec0dac8c P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.066730 15476 tablet_replica.cc:333] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.067545 15476 raft_consensus.cc:2243] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.068614 15476 raft_consensus.cc:2272] T be6c4adb5a2a4a34b6c89e15242b20a1 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.071413 15476 tablet_replica.cc:333] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.072281 15476 raft_consensus.cc:2243] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.072974 15476 raft_consensus.cc:2272] T 3326ab1787714e708abfea6cf0509109 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.075804 15476 tablet_replica.cc:333] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.076664 15476 raft_consensus.cc:2243] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.079154 15476 raft_consensus.cc:2272] T 81853b1e217c4b3aa8033dd7e0fdf211 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.082103 15476 tablet_replica.cc:333] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.082924 15476 raft_consensus.cc:2243] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.083745 15476 raft_consensus.cc:2272] T 3551e95f0ec5435f84de7b8f75069b13 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.084188 25092 raft_consensus.cc:493] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.084698 25092 raft_consensus.cc:515] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:41.086485 15476 tablet_replica.cc:333] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.087481 15476 raft_consensus.cc:2243] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.086812 25092 leader_election.cc:290] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.088708 15476 raft_consensus.cc:2272] T 3249701d37834e80bbe408e24ced7d54 P efa00eb5e7404570958fdebb6cd077e8 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.091688 15476 tablet_replica.cc:333] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.092594 15476 raft_consensus.cc:2243] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.093722 15476 raft_consensus.cc:2272] T 374e8578b72347509cc3ea1e6f464f31 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:41.095218 16708 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.096087 16708 leader_election.cc:336] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.096534 15476 tablet_replica.cc:333] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.096500 16708 leader_election.cc:304] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.097425 25110 raft_consensus.cc:2749] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:41.097500 15476 raft_consensus.cc:2243] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.098569 15476 raft_consensus.cc:2272] T 98936dee7edb4aa5873923a47699a89c P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.100849 15476 tablet_replica.cc:333] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.100822 25106 raft_consensus.cc:493] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.101434 25107 raft_consensus.cc:493] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.101609 25106 raft_consensus.cc:515] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.102021 15476 raft_consensus.cc:2243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.101976 25107 raft_consensus.cc:515] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.103302 15476 raft_consensus.cc:2272] T 4951b0fadf5b43e49a71f0bb4fb7a85e P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.104352 25107 leader_election.cc:290] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.106462 25106 leader_election.cc:290] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:41.106760 15476 tablet_replica.cc:333] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
W20260709 13:21:41.107800 17376 leader_election.cc:336] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:21:41.112265 17374 leader_election.cc:336] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.112630 15476 raft_consensus.cc:2243] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.112773 17374 leader_election.cc:304] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:41.113665 25106 raft_consensus.cc:2749] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:41.113849 17374 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
W20260709 13:21:41.114706 17374 leader_election.cc:336] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:41.115010 15476 raft_consensus.cc:2272] T b20d43e167554c06a6ae722c1a39c5d3 P efa00eb5e7404570958fdebb6cd077e8 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:41.114916 17376 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.115234 25110 raft_consensus.cc:493] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.117547 15476 tablet_replica.cc:333] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.118477 15476 raft_consensus.cc:2243] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.120232 15476 raft_consensus.cc:2272] T 9fe018a9ac6d466983da014c7b86d36f P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.118175 25110 raft_consensus.cc:515] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:41.122907 15476 tablet_replica.cc:333] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.122575 25110 leader_election.cc:290] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.123955 15476 raft_consensus.cc:2243] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.115185 17374 leader_election.cc:304] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.125201 25106 raft_consensus.cc:2749] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:41.126071 15476 raft_consensus.cc:2272] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:41.128886 16708 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:41.129122 15476 tablet_replica.cc:333] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
W20260709 13:21:41.129838 16708 leader_election.cc:336] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.130198 15476 raft_consensus.cc:2243] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.130278 16708 leader_election.cc:304] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.131223 15476 raft_consensus.cc:2272] T 73089175ee9c465197a1a11e304346be P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.131258 25110 raft_consensus.cc:2749] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:21:41.133749 15476 tablet_replica.cc:333] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.134411 15476 raft_consensus.cc:2243] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.134976 15476 raft_consensus.cc:2272] T ada6718f86ea49aa96bb0fdac1acca8e P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.137372 15476 tablet_replica.cc:333] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.137984 15476 raft_consensus.cc:2243] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.139021 15476 raft_consensus.cc:2272] T 0ecf0f1df23742a2a06e910e6b7350a0 P efa00eb5e7404570958fdebb6cd077e8 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.141587 15476 tablet_replica.cc:333] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.142285 15476 raft_consensus.cc:2243] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.143297 15476 raft_consensus.cc:2272] T 10731e2d513f466faf0d3f2030af5469 P efa00eb5e7404570958fdebb6cd077e8 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.145927 15476 tablet_replica.cc:333] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.146703 15476 raft_consensus.cc:2243] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:41.147022 17376 consensus_peers.cc:597] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.147637 15476 raft_consensus.cc:2272] T 173ca11ec288406aa68fe01a80d48e30 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.150125 15476 tablet_replica.cc:333] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.150956 15476 raft_consensus.cc:2243] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.152047 15476 raft_consensus.cc:2272] T ecb93faa15cf4f7b853b030dfdc6af28 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.154143 15476 tablet_replica.cc:333] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.154744 15476 raft_consensus.cc:2243] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.155366 15476 raft_consensus.cc:2272] T a931fb289f824ff28e993f8590e64b69 P efa00eb5e7404570958fdebb6cd077e8 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.157500 25110 raft_consensus.cc:493] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.157684 15476 tablet_replica.cc:333] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.158375 15476 raft_consensus.cc:2243] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.158008 25110 raft_consensus.cc:515] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } }
I20260709 13:21:41.160279 15476 raft_consensus.cc:2272] T 56335fb09b2846e184d61194e1d17f74 P efa00eb5e7404570958fdebb6cd077e8 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.160323 25110 leader_election.cc:290] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.162838 15476 tablet_replica.cc:333] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.163547 15476 raft_consensus.cc:2243] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.164139 15476 raft_consensus.cc:2272] T f836a80f84054393b6251b24eaf890f1 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.166502 15476 tablet_replica.cc:333] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
W20260709 13:21:41.166333 16708 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.167145 16708 leader_election.cc:336] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.167388 15476 raft_consensus.cc:2243] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.167575 16708 leader_election.cc:304] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efa00eb5e7404570958fdebb6cd077e8; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.168147 15476 raft_consensus.cc:2272] T e5e5fb284dac45b2a4c12e2ec38a1784 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.168828 25110 raft_consensus.cc:2749] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:41.170552 15476 tablet_replica.cc:333] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.171285 15476 raft_consensus.cc:2243] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.172395 15476 raft_consensus.cc:2272] T 40930ae2df144d709bfb017b86dfbf2b P efa00eb5e7404570958fdebb6cd077e8 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.174693 15476 tablet_replica.cc:333] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.175491 15476 raft_consensus.cc:2243] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.176120 15476 raft_consensus.cc:2272] T 5bfdfe1aa047409c9214ba1f43eb434a P efa00eb5e7404570958fdebb6cd077e8 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.178627 15476 tablet_replica.cc:333] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.179445 15476 raft_consensus.cc:2243] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.180078 15476 raft_consensus.cc:2272] T abd752e770ac43a799af500f16d446d5 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.180066 25106 raft_consensus.cc:493] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.180807 25106 raft_consensus.cc:515] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } attrs { replace: true } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.184111 15476 tablet_replica.cc:333] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.185024 15476 raft_consensus.cc:2243] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:21:41.187275 17376 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:21:41.188431 15476 raft_consensus.cc:2272] T d02ec9cc775248dfa8321ae141d0ad37 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.189277 25106 leader_election.cc:290] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:41.192472 15476 tablet_replica.cc:333] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
W20260709 13:21:41.193187 17374 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:41.193688 15476 raft_consensus.cc:2243] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.193894 17374 leader_election.cc:304] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:41.194969 15476 raft_consensus.cc:2272] T f71532fa5e704f9590387cd12f98666b P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.194948 25106 raft_consensus.cc:2749] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:21:41.196638 17374 leader_election.cc:336] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.197942 15476 tablet_replica.cc:333] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.198850 15476 raft_consensus.cc:2243] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.199999 15476 raft_consensus.cc:2272] T e0b4dc6fa23d44edb449d1be0842b184 P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.202687 15476 tablet_replica.cc:333] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.203584 15476 raft_consensus.cc:2243] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.204703 15476 raft_consensus.cc:2272] T 04d635b36e854e4f9ce8b973de8edafe P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.206776 25106 raft_consensus.cc:493] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.208063 15476 tablet_replica.cc:333] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.208938 15476 raft_consensus.cc:2243] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.209631 15476 raft_consensus.cc:2272] T 2125e4b55933488bbb6e25dba4189adb P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.212574 15476 tablet_replica.cc:333] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.213419 15476 raft_consensus.cc:2243] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:21:41.213690 17376 consensus_peers.cc:597] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.207639 25106 raft_consensus.cc:515] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.214797 15476 raft_consensus.cc:2272] T 8b778f817264440987b35333e6540c72 P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.216019 25106 leader_election.cc:290] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.218040 15476 tablet_replica.cc:333] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.219048 15476 raft_consensus.cc:2243] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.220324 15476 raft_consensus.cc:2272] T 79e87765622246cc907a538b35c52c64 P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.223297 15476 tablet_replica.cc:333] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.224217 15476 raft_consensus.cc:2243] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.225287 15476 raft_consensus.cc:2272] T ff94f30d9bba44ee9059993ecbbbf398 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:41.225638 17374 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.226807 17374 leader_election.cc:336] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.227367 17374 leader_election.cc:304] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.228111 25106 raft_consensus.cc:2749] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:21:41.228377 15476 tablet_replica.cc:333] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.229449 15476 raft_consensus.cc:2243] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.230710 15476 raft_consensus.cc:2272] T d9aa91533ea94ae392057ef582e0c419 P efa00eb5e7404570958fdebb6cd077e8 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.232811 15476 tablet_replica.cc:333] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.233384 15476 raft_consensus.cc:2243] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.234292 15476 raft_consensus.cc:2272] T d104b6d16beb411c9099f23901cebbb2 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.236213 15476 tablet_replica.cc:333] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.236734 15476 raft_consensus.cc:2243] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.237704 15476 raft_consensus.cc:2272] T fc0ed8d0a60640a0a5ce3a772c3eef5c P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.239871 15476 tablet_replica.cc:333] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.240526 15476 raft_consensus.cc:2243] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.241132 15476 raft_consensus.cc:2272] T cc977d4c44af4e80b58e4e0864085d9a P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:41.243116 17376 consensus_peers.cc:597] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.243497 15476 tablet_replica.cc:333] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.244189 15476 raft_consensus.cc:2243] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.244831 15476 raft_consensus.cc:2272] T 0445daac2da44dc5af84baa8514bb195 P efa00eb5e7404570958fdebb6cd077e8 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.246768 15476 tablet_replica.cc:333] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.247375 15476 raft_consensus.cc:2243] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.248175 15476 raft_consensus.cc:2272] T 3386b3f36f454834aaf359817c1adadb P efa00eb5e7404570958fdebb6cd077e8 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.250370 15476 tablet_replica.cc:333] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.251091 15476 raft_consensus.cc:2243] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.252146 15476 raft_consensus.cc:2272] T 92129e39cb38498ea9f0addc0a33f247 P efa00eb5e7404570958fdebb6cd077e8 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.254492 15476 tablet_replica.cc:333] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.255043 15476 raft_consensus.cc:2243] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.255827 15476 raft_consensus.cc:2272] T 9f202d4fdd58454a9c8e006ea35dbc46 P efa00eb5e7404570958fdebb6cd077e8 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.257158 25106 raft_consensus.cc:493] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.258159 15476 tablet_replica.cc:333] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.257763 25106 raft_consensus.cc:515] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.258970 15476 raft_consensus.cc:2243] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:21:41.260069 15476 raft_consensus.cc:2272] T 92d5dbc15e7c4691b7ef572175476c3d P efa00eb5e7404570958fdebb6cd077e8 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.260134 25106 leader_election.cc:290] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.262310 15476 tablet_replica.cc:333] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.263041 15476 raft_consensus.cc:2243] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.263578 15476 raft_consensus.cc:2272] T 6d92a8b6dd5a4c64b749d3505ca2033e P efa00eb5e7404570958fdebb6cd077e8 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.264359 25106 raft_consensus.cc:493] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
W20260709 13:21:41.265511 17374 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:41.266116 15476 tablet_replica.cc:333] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8: stopping tablet replica
I20260709 13:21:41.266485 25106 raft_consensus.cc:515] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.267081 15476 raft_consensus.cc:2243] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:41.267784 15476 raft_consensus.cc:2272] T 8243711a48fd4d8c85d79e7c10801c4f P efa00eb5e7404570958fdebb6cd077e8 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:41.268565 25107 raft_consensus.cc:493] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.268762 25106 leader_election.cc:290] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:41.268604 17374 leader_election.cc:336] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.269416 17374 leader_election.cc:304] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.269244 25107 raft_consensus.cc:515] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.270943 25106 raft_consensus.cc:2749] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:21:41.271463 25107 leader_election.cc:290] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:41.276760 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.277312 17374 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.277999 17374 leader_election.cc:336] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.278385 17374 leader_election.cc:304] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.279124 25107 raft_consensus.cc:2749] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:21:41.279255 17374 leader_election.cc:336] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.279789 17374 leader_election.cc:304] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.280583 25107 raft_consensus.cc:2749] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:41.290313 25107 raft_consensus.cc:493] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.290920 25107 raft_consensus.cc:515] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.293365 25107 leader_election.cc:290] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:41.298130 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.301537 17374 leader_election.cc:336] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.302021 17374 leader_election.cc:304] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.302807 25107 raft_consensus.cc:2749] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:21:41.347695 17376 connection.cc:573] client connection to 127.15.29.3:36655 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:21:41.351136 17376 consensus_peers.cc:597] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 -> Peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Couldn't send request to peer efa00eb5e7404570958fdebb6cd077e8. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:21:41.362823 25107 raft_consensus.cc:493] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.363440 25107 raft_consensus.cc:515] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.366552 25107 leader_election.cc:290] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
I20260709 13:21:41.370393 25107 raft_consensus.cc:493] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.371047 25107 raft_consensus.cc:515] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
W20260709 13:21:41.371958 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:41.373329 25107 leader_election.cc:290] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:41.378619 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.379323 17374 leader_election.cc:336] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.379720 17374 leader_election.cc:304] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.380460 25107 raft_consensus.cc:2749] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:21:41.380595 17374 leader_election.cc:336] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.381229 17374 leader_election.cc:304] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.381953 25107 raft_consensus.cc:2749] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:21:41.391189 25107 raft_consensus.cc:493] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.391778 25107 raft_consensus.cc:515] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.394430 25107 leader_election.cc:290] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:41.399930 17376 leader_election.cc:336] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
W20260709 13:21:41.406836 17374 leader_election.cc:336] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.407430 17374 leader_election.cc:304] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:41.408324 25107 raft_consensus.cc:2749] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:21:41.447404 25107 raft_consensus.cc:493] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 170921ba08b74a33995ae7c7686f8109)
I20260709 13:21:41.447914 25107 raft_consensus.cc:515] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:41.450599 25107 leader_election.cc:290] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639)
W20260709 13:21:41.457276 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:41.458500 17374 leader_election.cc:336] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Network error: Client connection negotiation failed: client connection to 127.15.29.2:42639: connect: Connection refused (error 111)
I20260709 13:21:41.458917 17374 leader_election.cc:304] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, 3c3ca2a0390b46bb9a5eda03a64eda1a
I20260709 13:21:41.459628 25107 raft_consensus.cc:2749] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:21:41.462586 17374 consensus_peers.cc:597] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:21:42.117516 25107 raft_consensus.cc:493] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.118139 25107 raft_consensus.cc:515] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.122200 25107 leader_election.cc:290] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.124862 17376 leader_election.cc:336] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.145159 25094 raft_consensus.cc:493] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.145682 25107 raft_consensus.cc:493] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.145785 25094 raft_consensus.cc:515] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.146538 25107 raft_consensus.cc:515] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } attrs { replace: true } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.148411 25094 leader_election.cc:290] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:42.150899 25107 leader_election.cc:290] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.153558 17376 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
W20260709 13:21:42.157572 17376 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
W20260709 13:21:42.158510 17374 leader_election.cc:336] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:42.158998 17374 leader_election.cc:304] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:42.159698 25107 raft_consensus.cc:2749] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:21:42.159838 17374 leader_election.cc:336] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
I20260709 13:21:42.160382 17374 leader_election.cc:304] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:42.161055 25107 raft_consensus.cc:2749] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:21:42.173616 15476 tablet_server.cc:196] TabletServer@127.15.29.3:0 shutdown complete.
W20260709 13:21:42.207695 17374 consensus_peers.cc:597] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:21:42.223333 25107 raft_consensus.cc:493] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.224133 25107 raft_consensus.cc:515] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.227640 25107 leader_election.cc:290] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.233134 17376 leader_election.cc:336] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.249547 25107 raft_consensus.cc:493] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.250223 25107 raft_consensus.cc:515] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.253485 25107 leader_election.cc:290] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.258283 17376 leader_election.cc:336] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.273031 25107 raft_consensus.cc:493] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.273694 25107 raft_consensus.cc:515] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.276691 25107 leader_election.cc:290] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.288769 17376 leader_election.cc:336] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.292788 15476 tablet_server.cc:179] TabletServer@127.15.29.4:0 shutting down...
I20260709 13:21:42.298198 25107 raft_consensus.cc:493] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.299036 25107 raft_consensus.cc:515] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.299715 25094 raft_consensus.cc:493] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.300360 25094 raft_consensus.cc:515] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.302001 25107 leader_election.cc:290] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:42.302794 25094 leader_election.cc:290] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.309744 17374 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:42.310474 17374 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:42.313431 17376 leader_election.cc:336] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.315160 17376 leader_election.cc:304] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:42.316172 25094 raft_consensus.cc:2749] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:21:42.316363 17376 leader_election.cc:336] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.316977 17376 leader_election.cc:304] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:42.317816 25094 raft_consensus.cc:2749] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:21:42.329639 17374 consensus_peers.cc:597] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:21:42.353855 17374 consensus_peers.cc:597] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 -> Peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Couldn't send request to peer 3c3ca2a0390b46bb9a5eda03a64eda1a. Status: Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:21:42.367048 15476 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:21:42.368615 15476 tablet_replica.cc:333] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.369499 15476 raft_consensus.cc:2243] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.370347 15476 raft_consensus.cc:2272] T 3551e95f0ec5435f84de7b8f75069b13 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.372942 15476 tablet_replica.cc:333] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.373584 15476 raft_consensus.cc:2243] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.374357 15476 raft_consensus.cc:2272] T 81853b1e217c4b3aa8033dd7e0fdf211 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.376586 15476 tablet_replica.cc:333] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.377226 15476 raft_consensus.cc:2243] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.377805 15476 raft_consensus.cc:2272] T 894a0fdf94344f46b0426efbd37a6c40 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.380203 15476 tablet_replica.cc:333] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.380899 15476 raft_consensus.cc:2243] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.381407 15476 raft_consensus.cc:2272] T 8b778f817264440987b35333e6540c72 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.383771 15476 tablet_replica.cc:333] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.384385 15476 raft_consensus.cc:2243] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.385088 15476 raft_consensus.cc:2272] T e8b0d179ad714b59b34ab1dc57a57c02 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.387470 15476 tablet_replica.cc:333] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.388183 15476 raft_consensus.cc:2243] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.388710 15476 raft_consensus.cc:2272] T 0ecf0f1df23742a2a06e910e6b7350a0 P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.391096 15476 tablet_replica.cc:333] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.391724 15476 raft_consensus.cc:2243] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.392591 15476 raft_consensus.cc:2272] T b6b98cb4c3414b4e86be2fc2bcd9e2b5 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.395129 15476 tablet_replica.cc:333] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.395823 15476 raft_consensus.cc:2243] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.396529 15476 raft_consensus.cc:2272] T d9aa91533ea94ae392057ef582e0c419 P ba2df6a668144022a223e504ae571f21 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.399173 15476 tablet_replica.cc:333] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.399854 15476 raft_consensus.cc:2243] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.400807 15476 raft_consensus.cc:2272] T b40781eb2d304f9b86f5061ae572e1c7 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.403277 15476 tablet_replica.cc:333] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.403910 15476 raft_consensus.cc:2243] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.404397 15476 raft_consensus.cc:2272] T f1a71b94ecfb4155a5cfe591077e9a1c P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.406770 15476 tablet_replica.cc:333] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.407496 15476 raft_consensus.cc:2243] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.408663 15476 raft_consensus.cc:2272] T 73089175ee9c465197a1a11e304346be P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.410817 15476 tablet_replica.cc:333] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.411465 15476 raft_consensus.cc:2243] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.412184 15476 raft_consensus.cc:2272] T ada6718f86ea49aa96bb0fdac1acca8e P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.414235 15476 tablet_replica.cc:333] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.414813 15476 raft_consensus.cc:2243] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.415512 15476 raft_consensus.cc:2272] T e0b4dc6fa23d44edb449d1be0842b184 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.417812 15476 tablet_replica.cc:333] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.418399 15476 raft_consensus.cc:2243] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.419018 15476 raft_consensus.cc:2272] T f71532fa5e704f9590387cd12f98666b P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.421135 15476 tablet_replica.cc:333] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.421777 15476 raft_consensus.cc:2243] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.422281 15476 raft_consensus.cc:2272] T 98936dee7edb4aa5873923a47699a89c P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.424631 15476 tablet_replica.cc:333] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.425192 15476 raft_consensus.cc:2243] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 10 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.425958 15476 raft_consensus.cc:2272] T 3249701d37834e80bbe408e24ced7d54 P ba2df6a668144022a223e504ae571f21 [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.428001 15476 tablet_replica.cc:333] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.428591 15476 raft_consensus.cc:2243] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.429052 15476 raft_consensus.cc:2272] T be6c4adb5a2a4a34b6c89e15242b20a1 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.431600 15476 tablet_replica.cc:333] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.432301 15476 raft_consensus.cc:2243] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.432832 15476 raft_consensus.cc:2272] T 79e87765622246cc907a538b35c52c64 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.435518 15476 tablet_replica.cc:333] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.436156 15476 raft_consensus.cc:2243] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.436932 15476 raft_consensus.cc:2272] T bb360c38974b4352a4febd1e0aada45d P ba2df6a668144022a223e504ae571f21 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.439031 15476 tablet_replica.cc:333] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.439584 15476 raft_consensus.cc:2243] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.440047 15476 raft_consensus.cc:2272] T 92129e39cb38498ea9f0addc0a33f247 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.442101 15476 tablet_replica.cc:333] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.442704 15476 raft_consensus.cc:2243] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.443326 15476 raft_consensus.cc:2272] T 2dc3ee5aa5844e5e9c7e3f566e36a561 P ba2df6a668144022a223e504ae571f21 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.445420 15476 tablet_replica.cc:333] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.446144 15476 raft_consensus.cc:2243] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.446732 15476 raft_consensus.cc:2272] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.449368 15476 tablet_replica.cc:333] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.449961 15476 raft_consensus.cc:2243] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.450835 15476 raft_consensus.cc:2272] T 173ca11ec288406aa68fe01a80d48e30 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.453042 15476 tablet_replica.cc:333] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.453634 15476 raft_consensus.cc:2243] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.454180 15476 raft_consensus.cc:2272] T a931fb289f824ff28e993f8590e64b69 P ba2df6a668144022a223e504ae571f21 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.456569 15476 tablet_replica.cc:333] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.457156 15476 raft_consensus.cc:2243] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.457703 15476 raft_consensus.cc:2272] T ecb93faa15cf4f7b853b030dfdc6af28 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.459944 15476 tablet_replica.cc:333] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.460664 15476 raft_consensus.cc:2243] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.461431 15476 raft_consensus.cc:2272] T 4951b0fadf5b43e49a71f0bb4fb7a85e P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.463711 15476 tablet_replica.cc:333] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.464386 15476 raft_consensus.cc:2243] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.464535 25113 raft_consensus.cc:493] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.465116 15476 raft_consensus.cc:2272] T d02ec9cc775248dfa8321ae141d0ad37 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.465137 25113 raft_consensus.cc:515] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "170921ba08b74a33995ae7c7686f8109" member_type: VOTER last_known_addr { host: "127.15.29.2" port: 42639 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.467736 25113 leader_election.cc:290] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
I20260709 13:21:42.468325 15476 tablet_replica.cc:333] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.469239 15476 raft_consensus.cc:2243] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.470139 15476 raft_consensus.cc:2272] T 42839b9fe9ae47a5a548348ea9b4eee1 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:42.472512 17376 leader_election.cc:336] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.473238 15476 tablet_replica.cc:333] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.473879 15476 raft_consensus.cc:2243] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.474388 15476 raft_consensus.cc:2272] T 51e96ec0851c4732b9a3e1272e9fd80f P ba2df6a668144022a223e504ae571f21 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.476805 15476 tablet_replica.cc:333] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.477456 15476 raft_consensus.cc:2243] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.477989 15476 raft_consensus.cc:2272] T 446649301a1c45808dccfebfadfed54c P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.480157 15476 tablet_replica.cc:333] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.480708 15476 raft_consensus.cc:2243] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.481176 15476 raft_consensus.cc:2272] T 3386b3f36f454834aaf359817c1adadb P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.483224 15476 tablet_replica.cc:333] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.483779 15476 raft_consensus.cc:2243] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.484333 15476 raft_consensus.cc:2272] T f919f8e7484c4f359aedc6ac05b45076 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.486344 15476 tablet_replica.cc:333] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.486917 15476 raft_consensus.cc:2243] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.487602 15476 raft_consensus.cc:2272] T abd752e770ac43a799af500f16d446d5 P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.492704 15476 tablet_replica.cc:333] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.493256 15476 raft_consensus.cc:2243] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.494127 15476 raft_consensus.cc:2272] T bab84fe76da941dc98416ce3f8c6f35b P ba2df6a668144022a223e504ae571f21 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.496170 15476 tablet_replica.cc:333] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.496691 15476 raft_consensus.cc:2243] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.497180 15476 raft_consensus.cc:2272] T 7c74caaa7fe045f9a0f6ff3b3df97602 P ba2df6a668144022a223e504ae571f21 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.499161 15476 tablet_replica.cc:333] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.499648 15476 raft_consensus.cc:2243] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.500146 15476 raft_consensus.cc:2272] T 04d635b36e854e4f9ce8b973de8edafe P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.502043 15476 tablet_replica.cc:333] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.502579 15476 raft_consensus.cc:2243] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.503172 15476 raft_consensus.cc:2272] T 56335fb09b2846e184d61194e1d17f74 P ba2df6a668144022a223e504ae571f21 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.505446 15476 tablet_replica.cc:333] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.506105 15476 raft_consensus.cc:2243] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.507193 15476 raft_consensus.cc:2272] T ee99829f455541f09dac21015b00282e P ba2df6a668144022a223e504ae571f21 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.509341 15476 tablet_replica.cc:333] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.509888 15476 raft_consensus.cc:2243] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.510488 15476 raft_consensus.cc:2272] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.512467 15476 tablet_replica.cc:333] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.513006 15476 raft_consensus.cc:2243] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.513602 15476 raft_consensus.cc:2272] T 9b74fd3415d14ae3aeb38c349eca0494 P ba2df6a668144022a223e504ae571f21 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.515643 15476 tablet_replica.cc:333] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.516010 25113 raft_consensus.cc:493] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader efa00eb5e7404570958fdebb6cd077e8)
I20260709 13:21:42.516359 15476 raft_consensus.cc:2243] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.517012 15476 raft_consensus.cc:2272] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.516575 25113 raft_consensus.cc:515] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "3c3ca2a0390b46bb9a5eda03a64eda1a" member_type: VOTER last_known_addr { host: "127.15.29.1" port: 34889 } } peers { permanent_uuid: "efa00eb5e7404570958fdebb6cd077e8" member_type: VOTER last_known_addr { host: "127.15.29.3" port: 36655 } } peers { permanent_uuid: "ba2df6a668144022a223e504ae571f21" member_type: VOTER last_known_addr { host: "127.15.29.4" port: 41455 } attrs { promote: false } }
I20260709 13:21:42.520008 15476 tablet_replica.cc:333] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.520862 15476 raft_consensus.cc:2243] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:21:42.521544 15476 raft_consensus.cc:2272] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.521592 25113 leader_election.cc:290] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889), efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655)
W20260709 13:21:42.523927 17374 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 3c3ca2a0390b46bb9a5eda03a64eda1a (127.15.29.1:34889): Network error: Client connection negotiation failed: client connection to 127.15.29.1:34889: connect: Connection refused (error 111)
W20260709 13:21:42.525085 17376 leader_election.cc:336] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer efa00eb5e7404570958fdebb6cd077e8 (127.15.29.3:36655): Network error: Client connection negotiation failed: client connection to 127.15.29.3:36655: connect: Connection refused (error 111)
I20260709 13:21:42.525542 15476 tablet_replica.cc:333] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.525611 17376 leader_election.cc:304] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 3c3ca2a0390b46bb9a5eda03a64eda1a, efa00eb5e7404570958fdebb6cd077e8
I20260709 13:21:42.526486 15476 raft_consensus.cc:2243] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:21:42.526548 17376 raft_consensus.cc:2686] T d104b6d16beb411c9099f23901cebbb2 P ba2df6a668144022a223e504ae571f21: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:21:42.527216 15476 raft_consensus.cc:2272] T fc0ed8d0a60640a0a5ce3a772c3eef5c P ba2df6a668144022a223e504ae571f21 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:21:42.529560 15476 tablet_replica.cc:333] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21: stopping tablet replica
I20260709 13:21:42.530123 15476 raft_consensus.cc:2243] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:21:42.531090 15476 raft_consensus.cc:2272] T cc977d4c44af4e80b58e4e0864085d9a P ba2df6a668144022a223e504ae571f21 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:21:42.550756 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.561640 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.578770 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.588686 17376 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.633482 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.689710 17375 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.756158 17374 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.784869 17378 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:21:42.872658 25015 leader_election.cc:336] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Service unavailable: reactor is shutting down (error 108)
I20260709 13:21:42.873219 25015 leader_election.cc:304] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:42.873895 25015 raft_consensus.cc:2686] T 374e8578b72347509cc3ea1e6f464f31 P ba2df6a668144022a223e504ae571f21: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:21:42.877324 25015 leader_election.cc:336] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Service unavailable: reactor is shutting down (error 108)
I20260709 13:21:42.877884 25015 leader_election.cc:304] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:42.878548 25015 raft_consensus.cc:2686] T 04867b8a53e645f0b58c55ecec0dac8c P ba2df6a668144022a223e504ae571f21: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:21:42.880638 25015 leader_election.cc:336] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 170921ba08b74a33995ae7c7686f8109 (127.15.29.2:42639): Service unavailable: reactor is shutting down (error 108)
I20260709 13:21:42.881104 25015 leader_election.cc:304] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba2df6a668144022a223e504ae571f21; no voters: 170921ba08b74a33995ae7c7686f8109, efa00eb5e7404570958fdebb6cd077e8
W20260709 13:21:42.882319 25015 raft_consensus.cc:2686] T b20d43e167554c06a6ae722c1a39c5d3 P ba2df6a668144022a223e504ae571f21: unable to run election callback: Service unavailable: The pool has been shut down.
F20260709 13:21:42.884447 15476 leader_election.cc:206] Check failed: has_responded_ 
*** Check failure stack trace: ***
*** Aborted at 1783603302 (unix time) try "date -d @1783603302" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e800003c74) received by PID 15476 (TID 0x7f327872a840) from PID 15476; stack trace: ***
    @           0x2d8530 __tsan::CallUserSignalHandler() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
    @           0x2daa13 rtl_sigaction() at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
    @     0x7f327ffec980 (unknown) at ??:0
    @     0x7f32793eae87 gsignal at ??:0
    @     0x7f32793ec7f1 abort at ??:0
    @           0x2de526 abort at /apache/dev/git/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
    @     0x7f327c1bcd84 google::LogMessage::SendToLog() at ??:0
    @     0x7f327c1bd930 google::LogMessage::Flush() at ??:0
    @     0x7f327c1c2a6b google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f327fb90c9c kudu::consensus::LeaderElection::~LeaderElection() at ??:0
    @     0x7f327fb9e4df kudu::RefCountedThreadSafe<>::DeleteInternal() at ??:0
    @     0x7f327fb9e4aa kudu::DefaultRefCountedThreadSafeTraits<>::Destruct() at ??:0
    @     0x7f327fb9e466 kudu::RefCountedThreadSafe<>::Release() at ??:0
    @     0x7f327fb95f37 scoped_refptr<>::~scoped_refptr() at ??:0
    @     0x7f327fb922d3 kudu::consensus::LeaderElection::Run()::$_0::~Run() at ??:0
    @     0x7f327fb93eba std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7f327fb947da std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7f327fb93aab std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @           0x37d8bb std::__1::__function::__value_func<>::~__value_func() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:?
    @           0x3671da std::__1::function<>::~function() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2543
    @     0x7f327deb423e kudu::rpc::Proxy::RefreshDnsAndEnqueueRequest()::$_0::~RefreshDnsAndEnqueueRequest() at ??:0
    @     0x7f327deb55fa std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7f327deb5f2a std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7f327deb51cb std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @     0x7f32859fa64b std::__1::__function::__value_func<>::~__value_func() at ??:0
    @     0x7f32859cb6ea std::__1::function<>::~function() at ??:0
    @     0x7f327ca72f6b kudu::DnsResolver::RefreshAddressesAsync()::$_1::~RefreshAddressesAsync() at ??:0
    @     0x7f327ca7516a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
    @     0x7f327ca75ada std::__1::__function::__alloc_func<>::destroy() at ??:0
    @     0x7f327ca74d0b std::__1::__function::__func<>::destroy_deallocate() at ??:0
    @           0x37d8bb std::__1::__function::__value_func<>::~__value_func() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:?
    @           0x3671da std::__1::function<>::~function() at /apache/dev/git/kudu/thirdparty/installed/tsan/include/c++/v1/functional:2543
